Bugzilla – Full Text Bug Listing
| Summary: | Git migration request | ||
|---|---|---|---|
| Product: | Community | Reporter: | Ryan Schmitt <ryan.schmitt> |
| Component: | Git | Assignee: | Eclipse Webmaster <webmaster> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | andrew.m.finkbeiner, donald.g.dunne, jeff.c.phillips, michael.p.masterson, nobody, roberto.e.escobar, ryan.d.brooks |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
|
Description
Ryan Schmitt
We can set up an empty Git container for you to perform the migration yourself at any moment... Please see this document for more info:
http://wiki.eclipse.org/Git/Migrating_to_Git
Otherwise, we try to keep weekend work for emergencies as much as possible.
As for the time required ... I guess that depends on the side of your history, and how you want your Git space to be structured (one repo, multiple repos, etc).
Let me know how you'd like to proceed.
We plan to migrate the source to a single repository. Would the empty repository have to be manually configured such that all the committers have the right to push commits to the repository? Like with all our other Git repos, your project committers will be permitted to push commits that were done by themselves, but not by others. We do lift this restriction temporarily to allow you to import your history, however. Do I import the history by pushing my local git repository (which I already imported from SVN) to the new Eclipse.org git repository? > Do I import the history by pushing my local git repository (which I already
> imported from SVN) to the new Eclipse.org git repository?
You can do that, or you can use a shell on git.eclipse.org to import SVN->Git directly. Whichever you're comfortable with.
We can do the import for you if you'd prefer that -- just let me know.
We would like to take you up on your offer of doing the import for us. When you do the migration can you exclude the path _Retired/? Also when would be a good date we could plan to begin preventing all further SVN commits and when would the new GIT repository be ready after that? Thanks, Ryan Brooks (In reply to comment #6) > We would like to take you up on your offer of doing the import for us. When > you do the migration can you exclude the path _Retired/? I believe so. In any case, it can be filtered out afterwards. > Also when would be a good date we could plan to begin preventing all > further SVN commits and when would the new GIT repository be ready after that? I propose we do a few trials first, so that we can make sure it all goes well, then we can pick a date. I'll start something early next week so you and your team can have a look. Excellent! The trials will make it a smoother transition for us. Is your SVN repo very big? It seems like it's been importing for hours now. Yes. It has a non-trivial number of branches and is much bigger if the _Retired/ path is not excluded. I've done a first pass at the import. Go ahead and give it a try. If you want to try committing to it, I can erase the repo and re-import it again later. http://git.eclipse.org/c/osee/org.eclipse.osee.git/ For my own reference, here is what I used: build:$ svn2git -v --exclude _Retired svn://dev.eclipse.org/technology/org.eclipse.osee The above URL is the web view. Clone URIs are: Anonymous: git://git.eclipse.org/gitroot/osee/org.eclipse.osee.git http://git.eclipse.org/gitroot/osee/org.eclipse.osee.git Committer (push): ssh://git.eclipse.org/gitroot/osee/org.eclipse.osee.git I was able to successfully use the HTTP read-only access and the ssh access (including making a commit). However, another one of the committers gets an error using ssh, but HTTP works for him. He runs the command: git clone ssh://git.eclipse.org/gitroot/osee/org.eclipse.osee.git And gets the following error: Initialized empty Git repository in o:/org.eclipse.osee/.git/ ssh: git.eclipse.org: no address associated with name fatal: The remote end hung up unexpectedly His proxy is set and anyway http://git.eclipse.org/gitroot/osee/org.eclipse.osee.git works for him. Any ideas? Thanks, Ryan (In reply to comment #13) Another clue is that I now get the same error with git pull (which worked before): ssh: git.eclipse.org: no address associated with name fatal: The remote end hung up unexpectedly > He runs the command: > git clone ssh://git.eclipse.org/gitroot/osee/org.eclipse.osee.git The error message would seem to indicate a DNS name resolution failure for git.eclipse.org. I've tested our DNS and it all seems to work. Not many corporate proxies allow proxying SSH connections. You may want to discuss this with your IT team. We can also set up https for your repo(s) to allow secure commits/pushes for committers. If you use SSH, you may need to specify your Eclipse user id: git clone ssh://userid@git.eclipse.org/gitroot/osee/org.eclipse.osee.git ping git.eclipse.org also fails for him. I tried this morning and it works for me, but I'm at home using VPN. While I don't know the exact issue, I think you are right that it is a DNS name resolution failure on our end. We may need to use https instead of ssh. If you can confirm that you are unable to establish an SSH connection to git.eclipse.org (since the error indicates a name resolution problem) I'll set up https for you. The command "ssh rbrooks@git.eclipse.org" reports "ssh: git.eclipse.org: node name or service name not known" According to http://progit.org/2010/03/04/smart-http.html, Git 1.6.6 has very efficient support for http(s) unlike previous versions. What version of git is eclipse.org using? Again, it seems your computer doesn't even know how to reach git.eclipse.org ssh barney@blablabla.eclipse.org ssh: blablabla.eclipse.org: Name or service not known If you can, please run nslookup nslookup > git.eclipse.org Server: 192.168.0.1 Address: 192.168.0.1#53 Non-authoritative answer: Name: git.eclipse.org Address: 206.191.52.44 As an alternative, you can add the above IP to your /etc/hosts file, but I'd like to solve this DNS issue you seem to be experiencing. We had to do the following to get through our company proxy: 1. get corkscrew from http://www.agroman.net/corkscrew/ 2. compile corkscrew using cygwin - ./configure; make 3. copy corkscrew.exe into C:\Program Files\Git\bin 4. setup up an .ssh/config file with the following line: ProxyCommand /bin/corkscrew company.proxy.address company.proxy.port %h %p We are now able to clone the repository using ssh. > We had to do the following to get through our company proxy:
Roberto, I'm sorry -- I was convinced you had a DNS issue.
I can enable https for commits on your repo if you'd like. We're using 1.7.1 so we have the http smart server enabled.
Please confirm.
Yes, please.. :) I _think_ I've enabled https for commits on the osee Git repo. I'm a bit new at this, so can you kindly try it out and see if it works? If you clone via http you will not be able to push your local changes. It has to be https. Here is the result of cloning using https: $ git clone https://git.eclipse.org/gitroot/osee/org.eclipse.osee.git Initialized empty Git repository in c:/UserData/git_https/org.eclipse.osee/.git/ error: error setting certificate verify locations: CAfile: /bin/curl-ca-bundle.crt CApath: none while accessing https://git.eclipse.org/gitroot/osee/org.eclipse.osee.git/info/refs fatal: HTTP request failed This was my fault... I forgot to add a rule on our load balancer for git over https. It worked fine from my office since I sit 'behind' the load balancer. This leads me to believe git over https has never worked.. On a related note, you will need to specify your committer ID in the clone URI, or your subsequent pushes will fail. $ git clone https://rbrooks@git.eclipse.org/gitroot/osee/org.eclipse.osee.git I still get the same result with or without my user name: $ git clone https://rbrooks@git.eclipse.org/gitroot/osee/org.eclipse.osee.git Initialized empty Git repository in c:/UserData/git_https/org.eclipse.osee/.git/ Password: error: error setting certificate verify locations: CAfile: /bin/curl-ca-bundle.crt CApath: none while accessing https://rbrooks@git.eclipse.org/gitroot/osee/org.eclipse.osee.git/info/refs fatal: HTTP request failed When in doubt, search Lars' blog. Ryan, does this help: http://www.vogella.de/blog/2010/08/09/git-https/ On windows, I followed Lars suggestion and added the following to git config - git config --global http.sslcainfo \bin/curl-ca-bundle.crt I am happily cloning the repository now. Thanks Denis. Lars is a genius, there is no doubt about it. @Denis Thanks for the feedback, very kind. :-) When I attempt to push I get an error with hooks/update. I found a post online suggesting a non-blank description of the repository needs to be added to .git/description. $ git push Password: Counting objects: 23, done. Delta compression using up to 2 threads. Compressing objects: 100% (6/6), done. Writing objects: 100% (12/12), 810 bytes, done. Total 12 (delta 4), reused 0 (delta 0) remote: Can't call method "get_value" on an undefined value at hooks/update line 254. remote: error: hook declined to update refs/heads/master (removing Lars since he likely doesn't want to see future updates)
> When I attempt to push I get an error with hooks/update.
The problem in that our update hook exists to enforce accurately tracking the provenance in the commit log. We examine the user id who is performing the push and ensure that user is the Committer of all the commits within the push. This works well on SSH since the SSH daemon uses a *nix system auth module.
For https, the auth is performed by the webserver, so our hook is seeing the push performed as Apache.
I believe the http smart server sets the authenticated user id in an environment variable. I'll see if I can tweak the hook to examine that as well. Stay tuned.
The hook has been updated to use $ENV{REMOTE_USER} if the user performing the push is Apache. can you try your push now?
Please be aware that the push will fail if your committer id or committer email do not appear in any one commit you're trying to push. You can see this via:
git log --pretty=full
I was now able to successfully push the commit 9b488b7a5e454fc9b945f950d627e4d4bbd33a48 "improved formatting". We are ready to go live. When will you be able to support the official migration? Thanks, Ryan (In reply to comment #36) > We are ready to go live. When will you be able to support the official > migration? I can do it now. I can do it anytime tomorrow, from 10:00 to 4:00pm ET I can do it anytime next week. And the week after. Let me know when is best for you and your team. Now would be great. I could send an email to OSEE Developers mailing list announcing that the migration is underway and to stop all commits until you give us the word. I'd appreciate your doing so before I proceed. I'll sit here and wait for your return. BTW - we also agree that I will be destroying your current Git repo, and replacing it with a fresh import from CVS, right? I have sent the email to the OSEE Developers mailing list. Yes, please replace the current Git repository with a fresh import from SVN excluding the path _Retired Ryan, I apologize, but I lied -- I'm not ready. As it turns out, the maintenance we've done on our various servers has broken the svn2git routines that worked back in comment 11. I'm rebuilding all that now, but I won't be able to perform the import until tomorrow morning. After I've re-tested the import to make sure it works, I'll report back here. Apologies for the setback. I've got all the tools compiled and working, and the import is happening now. Hi Denis, How is the migration going? Thanks, Ryan The import is complete, and everything should all be well. Please let me know how it works out. Denis I don't seem to be able to push any changes. Two examples are provided below. for reference "git config -l" includes the following settings: remote.origin.url=https://rbrooks@git.eclipse.org/gitroot user.name=rbrooks $ git push origin 0.9.5.v201008280938 Password: Counting objects: 1, done. Writing objects: 100% (1/1), 180 bytes, done. Total 1 (delta 0), reused 0 (delta 0) error: unpack failed: unpack-objects abnormal exit $ git push origin 0.9.6_dev Password: Counting objects: 448, done. Delta compression using up to 2 threads. Compressing objects: 100% (179/179), done. Writing objects: 100% (275/275), 24.81 KiB, done. Total 275 (delta 158), reused 0 (delta 0) error: unpack failed: index-pack abnormal exit I had forgotten to apply an ACL that allows the webserver to write to the git files. Grrr. I apologize -- I will get this right some day. Can you please try now? Both pushes from my previous comment worked this time. The next issue seems to be that (at least from the web interface http://git.eclipse.org/c/osee/org.eclipse.osee.git/) the Author is listed as "unknown" > The next issue seems to be that (at least from the web interface
> http://git.eclipse.org/c/osee/org.eclipse.osee.git/) the Author is listed as
> "unknown"
That is odd. The commit log looks fine, so I'll have to go poking into the cgit code to see what it's trying to do.
On a side note, do you mind if I run git-gc on your repository? This should compact and compress more aggressively, and I'm curious to see the results on a larger repo like yours. This should have no impact on anything other than a smaller clone size for future git clone operations.
We are in the process of deleting a few test branches and replacing old branches that had no changes with tags on master. You are welcome to run gc now, but we will want to run it again when we are done. Are we able to run gc, or will we just need to ask you to do it at that time? Anyone on your team who has SSH access can run git gc. Feel free to do this whenever you'd like. I can clone fine from windows using: git clone https://afinkbein@git.eclipse.org/gitroot/osee/org.eclipse.osee.git I am unable to clone from linux/solaris using the same command. Ryan Brooks was able to clone from my unix user account, using the same config, without issue. Any ideas on what could be going wrong? here is the error message: $ git clone https://afinkbein@git.eclipse.org/gitroot/osee/org.eclipse.osee.git Initialized empty Git repository in /somefolder/git/org.eclipse.osee/.git/ Password: error: The requested URL returned error: 401 while accessing https://afinkbein@git.eclipse.org/gitroot/osee/org.eclipse.osee. git/info/refs fatal: HTTP request failed I'm confident that I'm typing the correct password, since it works on windows and I've verified that if I type the wrong thing it fails. And I've tried retyping the password multiple times on both windows and unix to make sure I'm not hitting the wrong keys. Thanks, Andy Guys, do we have anything left to do here? Denis, We are happily using Git and it is working beautifully for us. The migration was flawless. Thanks for the great support. Regards, Ryan Ryan, I'm glad to read that -- I'll close this as 'fixed' then. Please open new bugs if any issues arise. |