Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 347885 - modified file reported as removed not staged
Summary: modified file reported as removed not staged
Status: RESOLVED DUPLICATE of bug 333338
Alias: None
Product: EGit
Classification: Technology
Component: Core (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 critical (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-01 02:52 EDT by François Rey CLA
Modified: 2011-06-07 19:39 EDT (History)
3 users (show)

See Also:


Attachments
Egit commit dialog showing file removed not staged (419.50 KB, image/jpeg)
2011-06-01 02:54 EDT, François Rey CLA
no flags Details
archive of the project in which the bug occurs (1013.15 KB, application/octet-stream)
2011-06-01 02:56 EDT, François Rey CLA
no flags Details
NOT A FIX: only adds a Status CLI command (7.24 KB, patch)
2011-06-04 11:08 EDT, François Rey CLA
no flags Details | Diff
Test projects to reproduct the bug (72.09 KB, application/octet-stream)
2011-06-04 15:34 EDT, François Rey CLA
no flags Details
Fix (6.33 KB, patch)
2011-06-05 18:59 EDT, François Rey CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description François Rey CLA 2011-06-01 02:52:21 EDT
See attached project archive.
The file egit/jgit does not see is tinymce.info. In Eclipse it's listed as removed, not staged, while in fact it's tracked and modified.
See attached project archive and screenshot.
Here's what the command line says:
[fr@m tinymce]$ git status
# On branch varin.info
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#       modified:   tinymce.info
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       .project
#       .settings/
#       tinymce.info~
#       tinymce/
no changes added to commit (use "git add" and/or "git commit -a")
[fr@m tinymce]$ git --version
git version 1.7.5.2

Running Eclipse 3.6.2 on Arch Linux 64 bit with:
Eclipse JGit (Incubation) 1.0.0.201105311521 org.eclipse.jgit.feature.group
Eclipse EGit (Incubation) 1.0.0.201105311611 org.eclipse.egit.feature.group
Comment 1 François Rey CLA 2011-06-01 02:54:35 EDT
Created attachment 197060 [details]
Egit commit dialog showing file removed not staged
Comment 2 François Rey CLA 2011-06-01 02:56:17 EDT
Created attachment 197061 [details]
archive of the project in which the bug occurs
Comment 3 François Rey CLA 2011-06-01 02:57:44 EDT
Note: Even though I used egit screenshot I reported this in jgit because I noticed this bug using jgit api.
Comment 4 François Rey CLA 2011-06-03 15:36:05 EDT
Changed to critical:
Other projects have been affected in my workspace, egit/jgit has become unreliable, and so have my scripts that depend on jgit.
Downgraded to 0.12.1 but problem is still there.
Comment 5 Chris Aniszczyk CLA 2011-06-03 17:52:15 EDT
I will try to reproduce.

If we have a test case, it makes it easier for us to reproduce and fix.
Comment 6 François Rey CLA 2011-06-03 18:38:31 EDT
I hope it can be reproduced. Somehow unpacking my project archive and importing it as new project did not reproduce the bug for me.
I tried disconnecting/reconnecting an affected project from eclipse but it did not help.
Deleting + importing + sharing also did not help.
Seeing this bug spreading to other projects in eclipse makes me wonder, maybe it's not just jgit but also egit.
Is there any project specific data kept by egit/jgit in eclipse that could have become corrupt?
Comment 7 François Rey CLA 2011-06-04 11:08:48 EDT
Created attachment 197345 [details]
NOT A FIX: only adds a Status CLI command

The added commands helps determine whether this is a jgit or egit issue.
Comment 8 François Rey CLA 2011-06-04 13:32:39 EDT
I carried out some investigation using a new jgit CLI Status command I created (patch attached) to see how jgit by itself would behave.

Conclusion:
The bug only appears when using jgit within eclipse, e.g. commit dialog, or in my case within a Groovy Monkey script loaded from a classloader within eclipse. Standalone 'jgit status' cli on affected projects remain consistent with native 'git status'.

Important clues/facts:
 1. affected projects correspond to modules of a larger project, each having its own project and git repository (no git submodules)
 2. in the filesystem main project and module projects are not overlapping because eclipse does not work well with such setup
 3. however at runtime modules must be within special subdirectories of the main project
 4. linked folders are used to recreate the required runtime structure (also helps developing/debugging/etc)
 5. jgit standalone outside eclipse on module projects show proper status
 6. jgit within eclipse on module projects show untracked files with a path starting from the main project, not the module
 
Fact #5 and #6 lead me to think there's something fishy going on with linked folders and jgit. It's as if jgit gets mixed up when the project (module) is the target of a linked folder within another project (the main project) that is also shared with egit. The main project git repository does not include or track the module, the latter is not even below the main project in the filesystem. However within eclipse jgit, which is supposed to be independent from eclipse, becomes 'aware' of the linked folder relationship which is an eclipse feature. The bug must be residing at this conjunction between jgit and eclipse resources.

I'm not sure how to take this further, but some more knowledgeable person can certainly help.

I do think it's an important bug to resolve for 1.0 release because such setup with linked folder is a well accepted scenario, Eclipse help even says:
"You can even use linked resources to overlap other resources in the workspace, so resources from one project can appear in another project".
Another interesting extract from eclipse help:
"The second mechanism for preventing linked resource creation is the team hook. If you define your own repository implementation, you can make use of the org.eclipse.core.resources.teamHook extension point to prevent the creation of linked resources in projects that are shared with your repository type. By default, repository providers do not allow linked resources in projects connected to the repository".
Clearly egit does not prevent linked resources, and I'm glad it does not.

Thanks for looking into this.
Comment 9 François Rey CLA 2011-06-04 15:34:52 EDT
Created attachment 197348 [details]
Test projects to reproduct the bug

I can confirm this bug is linked to linked folders.
I was able to reproduce it with simple projects in my workspace as follows:
-create a main project and add a few files
-init git repo, commit some files, and then modify and remove some, add to index some, so that the icons in package explorer shows the statuses
-the main project should be prefixed with > showing the changes
-duplicate this project into another identical project, and rename to 'module' for example
-import duplicated project and share with git and observe the >
-create a linked folder in a subdir of the main project linking to the module project
-observe the > disappearing from the modile project
-module commit dialog shows some additional untracked file that do not correspond so anything in the module project
Comment 10 François Rey CLA 2011-06-04 16:00:01 EDT
Note: the previously attached test-projects.zip contains projects that need to be imported. If while importing you do not copy the projects into your workspace (and leave them inside /extraction/path) you need to set the main project BASE_DIR variable to the base directory where all projects are located (i.e. /extraction/path).
The test projects are main, module1, module2, and module3. All module projects are already linked into the main project within different subfolders.
If you remove one linked folder in the main project, you should see the > reappear before the corresponding module project.
If you relink to the module project, the > will disappear before the corresponding module project.
Comment 11 François Rey CLA 2011-06-05 07:12:32 EDT
FYI: I created a separate bug #348318 for the Status CLI command patch. Use patch attached to that other bug.
Comment 12 François Rey CLA 2011-06-05 07:15:21 EDT
Reopening, this is has been marked as RESOLVED INVALID without my consent...
Second time it's happening to me when using Mylyn interface, what's the catch?
Comment 13 François Rey CLA 2011-06-05 18:39:10 EDT
Ok I can confirm this is due to the lack of support for linked resources in egit.
The iterator provided to jgit by egit is not aware of linked resources.
Changing to egit product.
Comment 14 François Rey CLA 2011-06-05 18:59:05 EDT
Created attachment 197368 [details]
Fix

Simple patch that ensure linked folders are ignored in key points of egit core and ui logic. It fixes the problem that originally made me file this bug.

While resolving this bug I also made a few changes to fix bug #327407, so another patch is attached there. These two patches should probably resolve a few bugs related to linked resources such as bug #333338 and bug #347454. They also solve the apparently unreported case where the EGit project sharing dialog lists repositories totally unrelated to the selected project when it contains linked folders pointing to files below another git repository.

Note that Eclipse docs state it is recommended for a Team
Provider to ignore linked resources, see section titled "Strategies for Handling Linked Resources" there:
http://help.eclipse.org/help30/index.jsp?topic=/org.eclipse.platform.doc.isv/guide/team_resources.htm
Comment 15 François Rey CLA 2011-06-07 19:39:00 EDT
I've submitted the following change request for this bug:
http://egit.eclipse.org/r/3680
It fixes this bug, plus cover a few more areas where linked resources should be ignored.
I'm marking this bug as duplicate of the more generic bug #333338.

*** This bug has been marked as a duplicate of bug 333338 ***