Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 434085 - "Follow Renames" in history view is still not working
Summary: "Follow Renames" in history view is still not working
Status: RESOLVED FIXED
Alias: None
Product: EGit
Classification: Technology
Component: Core (show other bugs)
Version: 3.2   Edit
Hardware: PC Windows 7
: P3 major with 7 votes (vote)
Target Milestone: 5.1   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-05-05 05:22 EDT by Timm H CLA
Modified: 2018-09-09 11:57 EDT (History)
10 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Timm H CLA 2014-05-05 05:22:01 EDT
The bug 302549 is resolved although the bug has not been fixed yet.

To reproduce:

- Rename a lot of files in a bigger project (think: renaming packages)
- Commit changes
- Select one file, 'Team' -> 'Show in History', make sure "Follow Renames" is enabled
- only the most recent commit (typically the rename/move) is shown
- open a console, enter 'git log --follow -- <FILENAME>' the complete history for the file is shown

The expected behaviour would be that "Follow Renames" produces the same results as "git log --follow". 

The bug 302549 mentions that this works via "Show Annotations" but i cannot reproduce this here.

Without having this issue resolved it is pretty much impossible to work with larger refactorings and Eclipse / EGit.
Comment 1 Robin Rosenberg CLA 2014-05-07 05:09:39 EDT
More hints are needed, i.e. it works here.

I renamed the whole src in org.eclipse.jgit twice and the history view could follow AddCommand.java

I also tried renaming the api package twice and the history could follow the AddCommand.java just fine.
Comment 2 Timm H CLA 2014-05-07 07:17:57 EDT
Okay, we moved companies and decided that, when switching from CVS to GIT (we imported the history using cvs2git), we are also going to rename our packages from

de.our.old.company

to

com.our.new.company

We are currently testing this with a repository that has approx. 1600 files, most of them are java files. We used an external script to move the files and update all references to the old packages (we do not only have them in java files but also on xml and other files, so eclipse wouldn't catch all those).

After commit both the moving and the editing of the files in the same commit, follow renames does not appear to work, while git log --follow does.

After I opened this bug we decided to try only moving the files and commiting then. After doing so Follow Renames seems to work as expected. 

This would be an OKish workaround for us (two commits instead of one) but this may still be something you might want to look into?
Comment 3 Robin Rosenberg CLA 2014-05-07 07:42:11 EDT
It doesn't matter that you did the change outside Eclipse. EGit does renaame detection.

There is a setting, diff.renamelimit whose default value is as low as 200. You
probably hit it. You can set it to a higher value and see it it helps. There is a cost to doing rename detection, so you may find it unbearably slow (or not).
Comment 4 Timm H CLA 2014-05-07 08:09:54 EDT
I tried setting this to 0 earlier which led to eclipse crashing / hanging when browsing the history. Most likely caused by the huge commit that moved and edtited 1000+ files.

This shouldn't really have any effect on this cause though since having it at the default, 'git log --follow' still works in both cases (move + edit in one commit, move and edit in seperate commits) while the EGit "Follow Renames" only gives the desired output in the later case.

The git-log manpage describes the '--follow' option as follows:

    Continue listing the history of a file beyond renames (works only for a single file).

The fact that it only works for a single files makes me assume that this option ignores any options like diff.renamelimit. I am not sure about this, though.

Is the '--follow' option implemented in JGit or EGit? If it is implement in JGit this bug maybe has to be moved there? If it is missing in JGit it should probably be added, although I don't know what the desired degree of compability is in JGit.
Comment 5 Robin Rosenberg CLA 2014-05-07 08:56:11 EDT
(In reply to Timm H from comment #4)
> I tried setting this to 0 earlier which led to eclipse crashing / hanging
> when browsing the history. Most likely caused by the huge commit that moved
> and edtited 1000+ files.

I guess the problem here is memory. Add more by editing the eclipse.ini file. Rename detection is a CPU and memory hog. Eclipse's default memory settings may be the main reason why the default limit is set to 200 (as opposed to 1000 for Git). You can check the workspace log file (workspace/.metadata/.log) to see if there are some clues. It might be that even given enough memory, Eclipse will be very slow with full rename detections enabled. 

> This shouldn't really have any effect on this cause though since having it
> at the default, 'git log --follow' still works in both cases (move + edit in
> one commit, move and edit in seperate commits) while the EGit "Follow
> Renames" only gives the desired output in the later case.
> 
> The git-log manpage describes the '--follow' option as follows:
> 
>     Continue listing the history of a file beyond renames (works only for a
> single file).
> 
> The fact that it only works for a single files makes me assume that this
> option ignores any options like diff.renamelimit. I am not sure about this,
> though.

Git has to do renamedetection first, which essentially means comparing all files with each other. I can't answer for sure, but I think Git has more advanced heuristics than JGit for limiting the number of compares and it packs data in memory better than Java can do.
 
> Is the '--follow' option implemented in JGit or EGit? If it is implement in
> JGit this bug maybe has to be moved there? If it is missing in JGit it
> should probably be added, although I don't know what the desired degree of
> compability is in JGit.

All Git logic is implemented in JGit. In this case there is a class called RenameDetector that does the actual work. EGit is just the UI.
I tried moving 1600 files and jgit log -M -p did detect the renames. JGit log -p -M -- path, strangely did not (bug), but I doubt that's the same issue that you are describing, despite its similarities.
Comment 6 Timm H CLA 2014-05-07 09:09:19 EDT
What I forgot to mention is, that this is a (multi-module-) Maven project and the artifiact ids were changed as well. So for example

de_our_company_project_a/src/main/java/de/our/company/project/a/File.java

was moved to 

com_new_company_project_a_b/src/main/java/com/new/company/project/a/b/File.java

(notice the extra directory "b")

Maybe this will help to reproduce.

Otherwise I think we can live with the workaround.

Another thing I noticed is, that in the history view you can't do compares between two revisions if the file was moved (and the rename is detected properly). I don't know if this is expected to work (using Compare with -> Commit...) but it would certainly be a nice feature :)
Comment 7 Robin Rosenberg CLA 2014-05-07 09:19:26 EDT
> Another thing I noticed is, that in the history view you can't do compares
> between two revisions if the file was moved (and the rename is detected
> properly). I don't know if this is expected to work (using Compare with ->
> Commit...) but it would certainly be a nice feature :)

Yea, it's a missing feature. There's another bug open for that
Comment 8 Robin Rosenberg CLA 2014-06-22 03:19:07 EDT
Did you try giving Eclipse more memory?
Comment 9 Henning Moll CLA 2015-01-27 17:42:03 EST
i had the same problem. After setting diff.renamelimit to a higher value, "follow" works as expected.

What about a message like 

"too many renames. You may want to increase your diff.renamelimit"

or

"too many renames. You may want to set your diff.renamelimit to at least XYZ"
Comment 10 Simon Schäfer CLA 2015-02-01 09:38:26 EST
I have the same problem as the TO. 'git log --follow <file>' works as expected, whereas in the Eclipse view after the first rename nothing else is shown (the follow renames feature is enabled).

One can see our history here: https://github.com/scala-ide/scala-ide/commits/master/org.scala-ide.sdt.core/src/org/scalaide/ui/internal/editor/ScalaSourceViewerConfiguration.scala

Github also does not follow renames. The first commit shown on github (https://github.com/scala-ide/scala-ide/commit/66c1c04f8873eaeb77caa2e07dd62b799e0156de) is also the commit where the history view stopped to work. It introduced a restructuring of our package structure.
Comment 11 Jakub Bocheński CLA 2016-05-23 07:21:31 EDT
> Without having this issue resolved it is pretty much impossible to work with larger refactorings and Eclipse / EGit.

what OP said..

Is a fix for this coming any time soon?
Comment 12 Dimo Petroff CLA 2018-01-19 10:44:24 EST
I have a similar problem. Follow Renames preference is on. The history shows a merge commit but nothing further. However, the rename is followed correctly in the "Show Revision Information" on the line numbers ruler, and when clicking on "Show in history" option from there, the merge magically changes into the correct commit in the history view. When shifting focus back to the editor, the correct commit is replaced once again with the merge commit.

To reproduce: look at the history of https://github.com/dsldevkit/dsl-devkit/blob/852664f8b202d30e826c80b5668095de328b50a3/com.avaloq.tools.ddk.xtext.expression/src/com/avaloq/tools/ddk/xtext/expression/generator/CodeGenerationX.xtend
Comment 13 Garret Wilson CLA 2018-09-01 12:03:11 EDT
Follow renames is not working for me, either. We moved all our Java files in a project to a different package. Now the history for each files shows a single commit: the commit in which all the renames occurred.

The following works fine:

    git log --follow -P FooBar.java

It's broken in Eclipse.

Eclipse Java EE IDE for Web Developers.

Version: 2018-09 M3 (4.9.0M3)
Build id: 20180830-1829
Comment 14 Garret Wilson CLA 2018-09-01 12:04:06 EDT
Git integration for Eclipse 5.1.0.201808281540-m3
Comment 15 Garret Wilson CLA 2018-09-01 12:15:55 EDT
I use `Team > Show Revision Information` and the entire files shows that it was modified by the developer who made the rename. Ugh. That functionality is now useless. The developer who renamed the file wasn't even working on this project when this file went through years of revision.
Comment 16 Garret Wilson CLA 2018-09-01 12:25:14 EDT
In https://bugs.eclipse.org/bugs/show_bug.cgi?id=302549#c23 Dimitry Polivaev said:

> … jGit does not follow renames for commits with number of added files larger than 200 . This number is hard coded into implementation of jGit.

Seriously!? Is this of the source of the problem? Somebody hard-coded an arbitrary literal into the code to prevent following history if we rename our packages for any significant project?

If that's the problem, can someone get rid of that arbitrary limit?
Comment 17 Garret Wilson CLA 2018-09-01 12:32:43 EDT
Following Dimitry's suggestion at https://bugs.eclipse.org/bugs/show_bug.cgi?id=302549#c23, I used:

    git config diff.renameLimit 99999

Now if I turn on "Show All Branches and Tags" in addition to "Follow History", I can finally see history.

Can someone take out that arbitrary, hard-coded limit, please? These "magic numbers" should never be put into code in the first place.
Comment 18 Matthias Sohn CLA 2018-09-06 05:38:49 EDT
(In reply to Garret Wilson from comment #17)
> Following Dimitry's suggestion at
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=302549#c23, I used:
> 
>     git config diff.renameLimit 99999
> 
> Now if I turn on "Show All Branches and Tags" in addition to "Follow
> History", I can finally see history.
> 
> Can someone take out that arbitrary, hard-coded limit, please? These "magic
> numbers" should never be put into code in the first place.

This value is not hard-coded but a default which you can change by configuring diff.renameLimit in git config.

Native git uses a default of 400 [1] whereas JGit defaults to 200 [2].
We can change the default in JGit to match the one used in native git.
Using a very large default is not an option since complexity of searching files for renames is O(n^2).

[1] https://github.com/git/git/blob/master/diff.c#L36
[2] https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/master/org.eclipse.jgit/src/org/eclipse/jgit/diff/DiffConfig.java#84
Comment 19 Eclipse Genie CLA 2018-09-08 19:33:39 EDT
New Gerrit change created: https://git.eclipse.org/r/128970
Comment 20 Eclipse Genie CLA 2018-09-09 07:04:38 EDT
New Gerrit change created: https://git.eclipse.org/r/128982
Comment 21 Eclipse Genie CLA 2018-09-09 11:54:32 EDT
Gerrit change https://git.eclipse.org/r/128982 was merged to [stable-5.1].
Commit: http://git.eclipse.org/c/jgit/jgit.git/commit/?id=a253c39729a98d6fdad3beb961c62d11a2fd2dd4
Comment 22 Matthias Sohn CLA 2018-09-09 11:57:43 EDT
Default for diff.renameLimit is now 400 like in native git.
If you want to increase it you can do this in the git configuration