Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 350188 - Git migration request
Summary: Git migration request
Status: CLOSED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Git (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-23 16:40 EDT by Jesse McConnell CLA
Modified: 2011-07-20 14:14 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jesse McConnell CLA 2011-06-23 16:40:16 EDT
RT/Jetty

The jetty project is plotting a migration to git within the next couple/few weeks.  Following the suggested procedure we are opening this issue to help track the effort and to lay a bug in the webmaster's ear.

* migration timeline

We would tentatively like to have the switch happen on the 6th of June.

* mapping of current code to new Git repos

We have been discussing this issue on our bug following the migration: Bug #312313 and this is the lately breakdown:

jetty -> org.eclipse.jetty.git
jetty-parent -> org.eclipse.jetty.parent.git
jetty-toolchain -> org.eclipse.jetty.toolchain.git
eclipse -> org.eclipse.jetty.releng.git 
jetty-wtp -> org.eclipse.jetty.wtp.git
admin -> org.eclipse.jetty.admin.git

* your decision regarding existing code (archive or import)

Import

* A description for each of your repositories. 

org.eclipse.jetty.git - jetty core code repository
org.eclipse.jetty.parent.git - jetty parent pom
org.eclipse.jetty.toolchain.git - jetty toolchain/build helper repository
org.eclipse.jetty.releng.git - jetty eclipse release management repository
org.eclipse.jetty.wtp.git - jetty WTP plugin repository
org.eclipse.jetty.admin.git - jetty admin files, code formatting, iplogs, docuware

I will keep this bug updated with developments on our end.
Comment 1 Jesse McConnell CLA 2011-06-23 18:09:53 EDT
err July 6th
Comment 2 Eclipse Webmaster CLA 2011-06-24 10:04:28 EDT
Ok, I've created /gitroot/jetty and made you the owner, so you can start importing whenever is good for you.

Just let me know when to mark your old data as read-only.

-M.
Comment 3 Denis Roy CLA 2011-06-24 15:02:52 EDT
> Ok, I've created /gitroot/jetty and made you the owner, so you can start
> importing whenever is good for you.

Unless of course you'd prefer we do the import.
Comment 4 Jesse McConnell CLA 2011-06-24 15:05:56 EDT
Thanks Denis, we are still kicking around the specifics but I think we have a pretty good path forward.  I had been thinking of doing it the 6th but maybe we ought to just pull the trigger earlier.

I'll check with greg and see, I suspect we might just have you guys do it...does the breakdown above look reasonable to you?
Comment 5 Denis Roy CLA 2011-06-24 15:46:28 EDT
At first glance I think your breakdown looks good.  Other projects seem to be going the 'one-repo-per-buildable-unit' route, or something similar.

We can test-run some imports early next week for you to examine and test.  Then, when you're ready and you've advised all your committers to cease work on CVS, we can trash the test Git repos and do a final import -- after which we mark CVS as read-only.

Does that make any sense?
Comment 6 Jesse McConnell CLA 2011-06-24 15:48:56 EDT
well, we are talking svn but yes, if you can do a test import for it all for use to take it for a test drive that would be awesome..

so if you can do that then whenever you have the time feel free to do a test import of it based on what we have here

thanks!
Comment 7 Jesse McConnell CLA 2011-06-27 17:44:24 EDT
Denis,

Could you do a trial run of this at your convenience this week?  I think we are going to do one last release of 7.4.3 on svn sometime this week and then take the plunge....so a trial run like you offered would let us take it for a test drive before committing :)

cheers,
jesse
Comment 8 Denis Roy CLA 2011-06-27 19:30:10 EDT
Sure thing, Jesse.  I've flagged this for first thing tomorrow morning.
Comment 9 Denis Roy CLA 2011-06-28 13:23:50 EDT
I running some tests imports now... I'll keep you posted.
Comment 10 Denis Roy CLA 2011-06-29 15:24:57 EDT
I've got all the repos imported.  You can see them at http://git.eclipse.org/c/ under the jetty section.

Currently only git:// and git+ssh:// work.  If you absolutely need https access, let me know and I'll make it happen.  Feel free to play around with these repos, and test pushing in new code -- they are not 'connected' to SVN in any way.
Comment 11 Jesse McConnell CLA 2011-07-01 06:48:07 EDT
Denis, 

thanks so much for getting this test run for us...I am playing with it.

initial thoughts:

layout ought to be the following, adding on .project to the core jetty git repo

jetty -> org.eclipse.jetty.project.git
jetty-parent -> org.eclipse.jetty.parent.git
jetty-toolchain -> org.eclipse.jetty.toolchain.git
eclipse -> org.eclipse.jetty.releng.git 
jetty-wtp -> org.eclipse.jetty.wtp.git
admin -> org.eclipse.jetty.admin.git
Comment 12 Jesse McConnell CLA 2011-07-01 09:37:50 EDT
Denis, I don't think these imported correctly, the top level of them has trunk/tags/branches when it ought to be the level of trunk with the tags/branches as git branches, etc...
Comment 13 Jesse McConnell CLA 2011-07-05 09:10:30 EDT
Denis,

Quick ping to bump this up the stack of bugs your wading through :)

Also, are you guys comfortable doing the migration form svn to git or would you rather me prepare this repo and then push it up?

Regardless, we need what is there now smoked so either you can redo the migration or I can get the repo's setup to do it myself.  I was hoping you guys could do it since it takes forever to slurp down the whole repo using 'git svn' from here.

cheers
Comment 14 Denis Roy CLA 2011-07-05 10:30:30 EDT
Jesse, if you want to do it, please be my guest.  If you have SSH access, feel free to use build.eclipse.org -- that ought to be much faster than doing it over the wire.
Comment 15 Jesse McConnell CLA 2011-07-05 11:18:40 EDT
ok, I'll take care of it

can you smoke the repo's that are there now then?  I'll use the initrepo script to recreate them.  I am going to take a small repo that is not used frequently for a ride first and then I'll schedule the lockdown with the others.

cheers,
jesse
Comment 16 Denis Roy CLA 2011-07-05 11:21:46 EDT
They're all gone.  I've made you the owner of /gitroot/jetty, so you should be all set.
Comment 17 Jesse McConnell CLA 2011-07-05 14:35:41 EDT
Denis,

I have the org.eclise.jetty.parent.git repo setup and working, able to push a release with maven and everything.

Could you enable your commit hook in there and I'll test with that?  once I confirm that working I'll migrate a couple more projects that no one will commit on.

cheers,
jesse
Comment 18 Jesse McConnell CLA 2011-07-05 17:04:56 EDT
Denis,

I have them all setup and copied over now...including the primary jetty
repository.

I would like to have the commit hooks wired up and work with it a little bit
before we call it final.  All but the o.e.j.project.git repo have limited
people that commit to them so they are likely final unless something comes up
but I want to give the main project one a bit of testing before we seal the
deal.

so if you could wire up the commit hooks that would be much appreciated

thanks!
jesse
Comment 19 Denis Roy CLA 2011-07-05 20:42:50 EDT
I've installed the update hook in each repo.  I've also set file permissions so that directories are group writable (and suid) and files group writable.

From where I sit, you should be all set.
Comment 20 Jesse McConnell CLA 2011-07-06 09:05:24 EDT
Thanks, I'll give it a whirl today and let you know!
Comment 21 Jesse McConnell CLA 2011-07-06 12:57:22 EDT
Denis,

We are used to workflow that give us commit emails.  Do you have something in mind for emails when there is a push into the repository?

something like:

http://git.kernel.org/?p=git/git.git;a=blob;f=contrib/hooks/post-receive-email;h=60cbab65d3f8230be3041a13fac2fd9f9b3018d5;hb=HEAD

cheers,
jesse
Comment 22 Jesse McConnell CLA 2011-07-06 13:32:03 EDT
Denis, 

Can you set svn to read-only now?  I believe we are good to go with this.

I am pushing up the latest project repo now, all the others are still in place and setup.  I'll put the update hook in for you as well while I am working on it.

Just to confirm, this setting it read-only is not something that can not be undone should I have screwed the proverbial pooch, right? :)

cheers,
jesse
Comment 23 Jesse McConnell CLA 2011-07-07 14:58:54 EDT
denis, ping?
Comment 24 Denis Roy CLA 2011-07-07 15:16:09 EDT
Sorry Jesse I'm out of the office today and Matt is on holidays...  This is on my list of things to look at tomorrow.
Comment 25 Jesse McConnell CLA 2011-07-07 15:17:03 EDT
ok, vacation is good, go back to it :)

tomorrow will be great!
Comment 26 Denis Roy CLA 2011-07-08 09:37:07 EDT
> Can you set svn to read-only now?  I believe we are good to go with this.

/svnroot/rt/org.eclipse.jetty has been set up read-only.  It's simply file permissions, so we can 'undo' this at any time.
Comment 27 Jesse McConnell CLA 2011-07-08 09:40:41 EDT
Thanks Denis,

we have a bit of work to do when hugues is back from vacation for the releng repos in there, and the releng.git will be going away in favor of the split out releng repos in that directory...

I re-rolled the project.git repo but I think I got your update and that stuff in there as well

so the only really outstanding thing is that push email script I mentioned earlier in the issue, what are your thoughts on getting that in?
Comment 28 Jesse McConnell CLA 2011-07-11 18:14:49 EDT
denis, I'll close this issue once I hear from you on if we can use that push email script mentioned above in this issue

cheers
Comment 29 Eclipse Webmaster CLA 2011-07-12 11:47:40 EDT
I've updated the linked script to fit our setup, and put the hook in all of the jetty repos(minus the x bit).   Would you like to start with a specific repo, or do you want me to enable them all at once?


-M.
Comment 30 Jesse McConnell CLA 2011-07-12 11:53:38 EDT
go ahead and enable them all, I'll test it out and we can call it good :)
Comment 31 Jesse McConnell CLA 2011-07-12 11:54:57 EDT
it goes to jetty-commit@eclipse.org right?
Comment 32 Eclipse Webmaster CLA 2011-07-12 11:58:25 EDT
(In reply to comment #31)
> it goes to jetty-commit@eclipse.org right?

Yes.

Ok they're all on now.

-M.
Comment 33 Jesse McConnell CLA 2011-07-12 12:02:13 EDT
what sort of lag time do you expect, I haven't seen a mail yet and I have pushed a couple of test commits
Comment 35 Eclipse Webmaster CLA 2011-07-12 12:34:15 EDT
I think the issue was a missing a+x on the permissions.

-M.
Comment 36 Jesse McConnell CLA 2011-07-12 12:36:08 EDT
closing, just got the commit mail

thanks for all your help on this!
Comment 37 Michael Gorovoy CLA 2011-07-12 17:59:01 EDT
I'm getting a following error whenever I push my commits into org.eclipse.jetty.project.git repository.

$ vi README.txt 
$ git add README.txt
$ git commit -m "test commit"
[master 22c54b1] test commit
 1 files changed, 0 insertions(+), 1 deletions(-)

$ git push
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 287 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: hooks/post-receive: line 718: expr: command not found
remote: hooks/post-receive: line 92: expr: command not found
remote: hooks/post-receive: line 96: expr: command not found
remote: hooks/post-receive: line 232: cat: command not found
remote: hooks/post-receive: line 659: grep: command not found
remote: hooks/post-receive: line 251: cat: command not found
remote: sendmail: fatal: genie@eclipse.org(9491): No recipient addresses found in message header
To ssh://mgorovoy@git.eclipse.org/gitroot/jetty/org.eclipse.jetty.project.git
   97b41c8..22c54b1  master -> master
Comment 38 Jesse McConnell CLA 2011-07-12 18:01:58 EDT
As a side note, this works for me but not for michael
Comment 39 Jesse McConnell CLA 2011-07-13 11:12:24 EDT
apparently this script only works for users that have shell access, otherwise it errors out like michael mentioned
Comment 40 Jesse McConnell CLA 2011-07-13 12:32:06 EDT
if this takes much of your time go ahead and remove it, once jetty is mirrored at github we can look at options there to push out mail or some sort of feed notification

thanks for everything!
Comment 41 Jesse McConnell CLA 2011-07-13 18:53:20 EDT
also the post-receive script is currently root owned so I can't help you guys out and fix/remove :/

i am trying to help :)
Comment 42 Eclipse Webmaster CLA 2011-07-14 14:56:22 EDT
Ok, this should be fixed now.  The script just needed to know where to find things when the restricted path was present.

-M.
Comment 43 Jesse McConnell CLA 2011-07-14 17:31:58 EDT
closing since it should be resolved, thanks for all the help
Comment 44 Eike Stepper CLA 2011-07-20 14:14:00 EDT
(In reply to comment #12)
> Denis, I don't think these imported correctly, the top level of them has
> trunk/tags/branches when it ought to be the level of trunk with the
> tags/branches as git branches, etc...

Jesse, it seems I'm facing the same problem with my SVN->Git import, bug 351050 comment 12. Can you explain to me how you've solved it?