Comment 1 Linas G. 2010-10-21 10:11:05 EDT

I have a very similar problem.
I can't clone git repsitory because my eclipse workspace resides on remote mapped network drive. 
As Mathias mentioned above, I can clone repository if my workspace is on my local computer with Windows.


org.eclipse.jgit.errors.TransportException: Cannot move pack to \\XX.XX.XX.XX\www\devel\gitproject\.git\objects\pack\pack-15795afef2e750906977e6e19aeabd8ea1bc91f6.pack
	at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:286)
	at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:224)
	at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:210)
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:145)
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:880)
	at org.eclipse.egit.core.op.CloneOperation.doFetch(CloneOperation.java:201)
	at org.eclipse.egit.core.op.CloneOperation.run(CloneOperation.java:119)
	at org.eclipse.egit.ui.internal.clone.GitCloneWizard.executeCloneOperation(GitCloneWizard.java:249)
	at org.eclipse.egit.ui.internal.clone.GitCloneWizard.access$3(GitCloneWizard.java:242)
	at org.eclipse.egit.ui.internal.clone.GitCloneWizard$4.run(GitCloneWizard.java:223)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: java.io.IOException: Cannot move pack to \\XX.XX.XX.XX\www\devel\gitproject\.git\objects\pack\pack-15795afef2e750906977e6e19aeabd8ea1bc91f6.pack
	at org.eclipse.jgit.transport.IndexPack.renameAndOpenPack(IndexPack.java:1137)
	at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:648)
	at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:279)
	... 11 more


My system

Windows 7
Eclipse for PHP Developers
Build id: 20100218-1602
Egit 0.9.3
eclipse.buildId=unknown
java.version=1.6.0_21
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=lt_LT
Framework arguments:  -product org.eclipse.epp.package.php.product

Comment 2 Sasa Zivkov 2011-01-26 09:52:29 EST

I couldn't reproduce this issue when using Samba on linux for sharing files
and cloning from EGit on windows (XP in this case).
The version of the Samba server was 3.5.6.

However, I am not sure what the bug reporters (Kenny Scott and Linas G.) used
for file sharing.
@Kenny, @Linas: please provide more details on how do you share the files from linux to windows.

Comment 3 Kenny Scott 2011-01-26 17:44:23 EST

Hi,

It was using samba. I am now no longer using Vista (partition table got corrupted, long story) and am now using Windows 7, so I will be unable to recreate with Vista, I'm afraid. I could try with Windows 7 though.

Comment 4 Sasa Zivkov 2011-01-27 02:59:30 EST

At SAP we tried cloning from Windows 7 and the same samba server and it finished
without any errors.

Comment 5 Sasa Zivkov 2011-06-17 05:16:13 EDT

Nobody seems to be able to reproduce this issue any more.
Closing it now.

Comment 6 Marc Herbert 2011-09-01 09:51:12 EDT

Same error here:

org.eclipse.jgit.api.errors.JGitInternalException: Exception caught during execution of fetch command
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:145)
at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:169)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:116)
at org.eclipse.egit.core.op.CloneOperation.run(CloneOperation.java:142)
at org.eclipse.egit.ui.internal.clone.GitCloneWizard.executeCloneOperation(GitCloneWizard.java:292)
at org.eclipse.egit.ui.internal.clone.GitCloneWizard.access$3(GitCloneWizard.java:285)
at org.eclipse.egit.ui.internal.clone.GitCloneWizard$5.run(GitCloneWizard.java:266)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: org.eclipse.jgit.errors.TransportException: Cannot move pack to Z:\mherber2\NFSrepo\.git\objects\pack\pack-7d4e1c07e58c54a9770004f2661f4fd2b0a86bce.pack
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:291)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:229)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:216)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1062)
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:136)
... 7 more
Caused by: java.io.IOException: Cannot move pack to Z:\mherber2\NFSrepo\.git\objects\pack\pack-7d4e1c07e58c54a9770004f2661f4fd2b0a86bce.pack
at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.renameAndOpenPack(ObjectDirectoryPackParser.java:466)
at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:190)
at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:429)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:672)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:284)
... 13 more

Windows 7
Egit 1.0.0.201106090707-r

Comment 7 Marc Herbert 2011-09-01 10:16:35 EDT

Cygwin's git works perfectly in the exact same environment.

Please re-open.

If I clone using Cygwin's git and then add the existing repo in Egit I get similar errors later down the road:


org.eclipse.core.runtime.CoreException: Cannot move pack to Z:\mherber2\NFSrepo\.git\objects\pack\pack-91930f3cad8bd5fe827a8f7f3a697bf706e29ff6.pack
at org.eclipse.egit.ui.internal.fetch.FetchOperationUI.execute(FetchOperationUI.java:102)
at org.eclipse.egit.ui.internal.fetch.FetchOperationUI$1.run(FetchOperationUI.java:117)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: org.eclipse.jgit.errors.TransportException: Cannot move pack to Z:\mherber2\NFSrepo\.git\objects\pack\pack-91930f3cad8bd5fe827a8f7f3a697bf706e29ff6.pack
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:291)
at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:229)
at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:216)
at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:149)
at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:111)
at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1062)
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:136)
at org.eclipse.egit.core.op.FetchOperation.run(FetchOperation.java:128)
at org.eclipse.egit.ui.internal.fetch.FetchOperationUI.execute(FetchOperationUI.java:99)
... 2 more
Caused by: java.io.IOException: Cannot move pack to Z:\mherber2\NFSrepo\.git\objects\pack\pack-91930f3cad8bd5fe827a8f7f3a697bf706e29ff6.pack
at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.renameAndOpenPack(ObjectDirectoryPackParser.java:466)
at org.eclipse.jgit.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:190)
at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:429)
at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:672)
at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:284)
... 10 more

Comment 8 Marc Herbert 2011-09-01 10:17:29 EDT

This line was dropped from the last comment, sorry for the noise:

  Same problem if I use the full UNC instead of a mapped drive.

Comment 9 Marc Herbert 2011-09-01 11:41:12 EDT

Using a different file server everything just works. What kind of additional information should I provide and how?

Comment 10 Sasa Zivkov 2011-09-21 10:53:57 EDT

(In reply to comment #9)
> Using a different file server everything just works. What kind of additional
> information should I provide and how?

If you can reproduce the issue then it would be useful if you strace the samba server process and reproduce the issue. The output of strace may help diagnosing the issue.

You may also strace the samba process on the other system where the issue doesn't occur and compare the two strace outputs.

If you never used strace check [1] for an example on how to strace apache server. Tracing the samba server should be similar.


[1] http://linuxgazette.net/132/vishnu.html

Comment 11 Marc Herbert 2011-10-13 09:32:09 EDT

(In reply to comment #10)
> If you can reproduce the issue

I can reproduce 100%

> then it would be useful if you strace the samba
> server process and reproduce the issue. 

Unfortunately both samba servers are way beyond my control. Would you be interested in packet captures instead?

Comment 12 Sasa Zivkov 2011-10-17 09:16:02 EDT

(In reply to comment #11)
> (In reply to comment #10)
> > If you can reproduce the issue
> 
> I can reproduce 100%
> 
> > then it would be useful if you strace the samba
> > server process and reproduce the issue. 
> 
> Unfortunately both samba servers are way beyond my control. Would you be
> interested in packet captures instead?

It's not the same as strace and I don't know much about the CIFS protocol.

How about looking at Samba logs and trying to figure out what caused the error:
http://oreilly.com/openbook/samba/book/ch09_01.html

Without any access to the samba server(s) where the error can be reproduced
it is quite difficult to troubleshoot this issue.

Comment 13 Michal Fabry 2012-02-26 12:56:46 EST

Hi. I'm facing almost the same issue.

I use Windows 7 and have my workspace on Linux machine exported through samba. 

When I'm trying to clone any project it initially starts to doing something (I can see progress bar and "Receiving objects" and "Resolving deltas" with percentage goes to 100% and then at the and it throws me an error "Git repository clone failed" (Exception stack trace is attached).

I tried to debug it and found out that during initial phase it successfully created a project directory within my workspace also with ".git" as a subdirectory but nothing else and it had removed everything after that error at the end.

I tried to play with that and ran dirty bash script on my linux machine (where I have my workspace) which basically does "chmod -R 0777 ." in a loop. With this script running on background I was sometimes able to clone a repository successfully. Therefore I guess it has something to do with a combination samba+permissions. But it's still weird because it was able to create a project directory within my workspace without any problems.

Eclipse 3.7.1
EGit 1.3.0.201202151440-r

Samba strace output attached.
EGit exception attached.

Comment 14 Michal Fabry 2012-02-26 12:57:27 EST

Created attachment 211640 [details]
EGit exception

Comment 15 Michal Fabry 2012-02-26 12:57:47 EST

Created attachment 211641 [details]
Samba strace

Comment 16 Michal Fabry 2012-04-26 05:36:34 EDT

So. The problem was in jgit and because of "renameTo" function. It doesn't work properly on windows when trying to move file from a local filesystem to a remote filesystem (e.g. samba). From what I've found on the internet this is a known issue in Java. Therefore I'm not sure if the best idea is to fix this in jgit rather then in Java itself.

The solution is to add a fallback function when renameTo fails to:
1. copy a source file byte by byte to a destination file
2. delete the source file

I'm attaching also my patch so you can see what I mean.

Comment 17 Michal Fabry 2012-04-26 05:37:18 EDT

Created attachment 214588 [details]
Patch for renameTo.

Comment 18 Marc Herbert 2012-11-28 08:32:45 EST

(In reply to comment #16)
> From what I've found on the internet this
> is a known issue in Java. Therefore I'm not sure if the best idea is to fix
> this in jgit rather then in Java itself.

The API documentation basically says "this renameTo() method is broken", so it is very unlikely to ever be fixed in java.

http://stackoverflow.com/questions/1000183/reliable-file-renameto-alternative-on-windows

(of course one really wonders why they provided this method in the first place; if it's just an optimization then they should have hidden it)

Michal's patch looks like the right approach: try to use the optimized method and fallback on a slower way if it fails.

At the very least the *reporting* of this error should urgently be made clearer; something along the lines of: "you have tried to use an unsupported filesystem, sorry don't do that".

Comment 19 Marc Herbert 2012-11-28 08:39:59 EST

Reproduced the exact same with 2.1.0.201209190230-r

BTW the bug could definitely use the "network drive" keyword in its title.

Comment 20 Nico Wagner 2013-01-09 10:03:35 EST

reproduced with:
- Eclipse EGit 2.1.0.201209190230-r
- Samba Version 3.2.5
- Windows 7

Comment 21 Robin Stocker 2013-02-25 08:32:50 EST

The fix from Michal goes in the right direction, just some comments:

* I think the code should be moved to a FileUtils#rename method and do the fallback there
* There are other places which use renameTo, they should probably also be replaced
* The chances of this getting reviewed is much higher if this was submitted to Gerrit, see http://wiki.eclipse.org/EGit/Contributor_Guide#Using_Gerrit_at_https:.2F.2Fgit.eclipse.org.2Fr

Comment 22 Matthias Sohn 2013-04-05 19:19:58 EDT

I think this fallback is too dangerous to be used since JGit relies on atomicity of rename to implement locking, the proposed fallback surely does not provide an atomic rename implementation. 

Hence I vote against the proposed patch. 

Instead we should improve propagation of the original error message so that users can easily see that they try to use JGit on a not supported file system combination.

Comment 23 Marc Herbert 2013-04-06 05:42:15 EDT

(In reply to comment #22)
> JGit relies on atomicity of rename to implement locking,...

But quoting the official renameTo() Javadoc:

"The rename operation might not be able to move a file from one filesystem to another, it might not be atomic, and it might not succeed if..."


I think this highlights a serious design bug. The severity should probably be increased. Samba looks like it's just the tip of the iceberg.


> Instead we should improve propagation of the original error message so that
> users can easily see that they try to use JGit on a not supported file
> system combination.

I think that's the very, very least that should be done.


A more proper fix would be to stop using the "best effort" renameTo() function at all and switch to an actually atomic and reliable replacement. Note this replacement would not need to be a general purpose rename function; e.g. it could for instance be limited to a single file and single filesystem if that's enough for JGit (I'm just guessing).

An alternative fix would be to have white and blacklists of filesystems where renameTo() can be trusted, and refuse to run at all where it cannot + a scary warning on unknown/unlisted FS.

A final idea would be to dynamically run a few renameTo() sanity checks before trying to create a new clone and stop immediately if they fail. This sounds a bit brittle though: the sanity checks better have rock-solid coverage.


I know Git was clearly not designed with network filesystems in mind, but:

1. In many big companies, network filesystems is where ALL work is supposed to happen (for a number of usual and obvious reasons)
2. In cases where it can't work then JGit should not even start running; otherwise the risk of data loss is too high.
3. According to the Java API this problem is NOT even restricted to network filesystems anyway.


Out of curiosity, I wonder if a similar atomic rename problem could affect non-Java clients and if yes how they handle it.

Comment 24 Marc Herbert 2013-04-06 05:46:50 EDT

(In reply to comment #23)
> 
> A final idea would be to dynamically run a few renameTo() sanity checks
> before trying to create a new clone and stop immediately if they fail. 

The above would not be enough: such checks would have to be run _not just before cloning but pretty much all the time_ since someone could create a clone on one working filesystem and then move it to non-working one with a plain operating system copy.

[Sorry for the noise]

Comment 25 Alfred Nathaniel 2013-04-09 04:30:24 EDT

Any date set when the minimum JVM requirement will be upped to 1.7?  Then java.nio.file.Files.move could be used.

Comment 26 Robin Rosenberg 2013-05-11 16:32:06 EDT

*** Bug 377736 has been marked as a duplicate of this bug. ***

Comment 27 Leigh Bicknell 2015-03-19 05:16:35 EDT

I also get this on Vista (With Egit 3.7). An almost identical error also occurs when doing a a pull or fetch from a Remote network (ssh) to a mounted Linux Samba share. Which makes sense having read the comments to this and its related bugs and glanced briefly at the code.

What is surprising is that nothing has been done to work around this as a temporary measure until a proper rework can be done. Or even at minimum a clearer error message in these cases would prevent users wasting time debugging Samba etc.

Is there any word on progress for this issue?

Comment 28 Eclipse Genie 2015-12-30 18:12:49 EST

Gerrit change https://git.eclipse.org/r/46469 was merged to [master].
Commit: http://git.eclipse.org/c/jgit/jgit.git/commit/?id=3fc93f8a562e96d354546ad6ff8c9dd56709c5e5

Comment 29 Andrey Loskutov 2015-12-30 18:22:49 EST

(In reply to Eclipse Genie from comment #28)
> Gerrit change https://git.eclipse.org/r/46469 was merged to [master].
> Commit:
> http://git.eclipse.org/c/jgit/jgit.git/commit/
> ?id=3fc93f8a562e96d354546ad6ff8c9dd56709c5e5

Latest nightly of jgit contains the possible fix. Please grab it and verify the fix (I have right now no access to NFS to test).

Comment 30 Matthias Sohn 2016-01-07 02:22:27 EST

fix was submitted

Comment 31 Thomas Wolf 2017-09-01 11:24:46 EDT

*** Bug 365055 has been marked as a duplicate of this bug. ***