Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 347151

Summary: egit fetch factory does only fetch, not merge
Product: [Technology] EGit Reporter: Matthias Kempka <mkempka>
Component: RelengAssignee: Project Inbox <egit.releng-inbox>
Status: NEW --- QA Contact:
Severity: normal    
Priority: P3 CC: aniefer, david_williams, gunnar, kim.moir, mknauer, nfalco79, pwebster, remy.suen, stephan.herrmann
Version: 0.10.0   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:

Description Matthias Kempka CLA 2011-05-25 09:33:27 EDT
Build Identifier: unit id='org.eclipse.egit.fetchfactory.feature.jar' version='0.10.0.201011091645'

I am working on doing PDE builds from git repositories in various environments, and everywhere it seems to me that updates, although they are fetched, are not merged into the repository when setting the ${fetchCacheLocation}. If the source repository contains updates, the output of my build generally looks like this:

GitUpdateLocalRepo:
     [echo] [GIT] /home/build/.hudson/jobs/fasttrack/workspace/fasttrack.source/ssh___cvs_eclipsesource_com_home_mkempka_gitrepos_fasttrack_source_git_ >> git fetch
     [exec] From ssh://<server>/home/mkempka/gitrepos/fasttrack.source
     [exec]    d69063d..c86f5f2  master     -> origin/master

GitCheckSkipClone:

GitCloneRepoToLocalRepo:

GitCheckoutTagInLocalRepo:
     [echo] [GIT] /home/build/.hudson/jobs/fasttrack/workspace/fasttrack.source/ssh___cvs_eclipsesource_com_home_mkempka_gitrepos_fasttrack_source_git_ >> git checkout --force master
     [exec] Already on 'master'
     [exec] Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.

GitFetchElementFromLocalRepo:
     [copy] Copying 65 files to /home/build/.hudson/jobs/fasttrack/workspace/builds/plugins/com.yoxos.launcher
 

Reproducible: Always

Steps to Reproduce:
1. Set up a simple feature build with a git repository and the egit fetchfactory
2. build once
3. commit an update to the git repository
4. try to execute a build with the update and see that it is not included.
Comment 1 Andrew Niefer CLA 2011-08-12 11:36:59 EDT
This would be a problem for any branch.  Workaround is to have a new fetchCacheLocation every build so the repo has everything from the initial clone.

To fix I think we need to identify whether or not the tag is actually branch or not.  To do this, I think we can run "git branch" after the checkout and see if the ${tag} is listed there, if it is, then after the checkout we should do a "git merge origin"
Comment 2 Nikolas Falco CLA 2011-08-17 04:13:32 EDT
Initially I had same problem.
Now I specify branch using "origin/" as prefix, so every times lastest revisions from branch are downloaded without a branch, execution example:

GitCheckoutTagInLocalRepo:
     [echo] [GIT] /home/builder/scmCache/ssh___rad_xxxxxx >> git checkout --force origin/4.6.v06
     [exec] Previous HEAD position was e620597... xxxx
     [exec] HEAD is now at b098fb2... CQI/0042

If you specify a tag from any branch not prefixes are needed:

GitCheckoutTagInLocalRepo:
     [echo] [GIT] /home/builder/scmCache/ssh___rad_yyyyy >> git checkout --force 4.6.10.v06-001.001.021
     [exec] HEAD is now at 1bfac55... CQI
Comment 3 Nikolas Falco CLA 2011-08-17 04:15:49 EDT
little mistake:

...revisions from branch are downloaded without CREATE a NEW LOCAL* branch ...
(that need merge next time)
Comment 4 Matthias Kempka CLA 2011-08-17 04:38:08 EDT
Nicolas, you are right with the origin/ prefix, but this is what the bug report is about.

If the local repository is cloned initially and the local branches are there just like in the developers workspace, I expect that I can adress them just like in the developers workspace. 

This means after pulling the changes they should be available in the local branches. If merging fails, it's ok to fail.

Local branches should be available, because the concept of origin/ is unknown to most young git developers and left out of most git introduction documentation. Relying on origin/ branches will effectively inhibit the use of this tool.
Comment 5 Gunnar Wagenknecht CLA 2012-01-27 04:12:49 EST
(In reply to comment #4)
> Local branches should be available, because the concept of origin/ is unknown
> to most young git developers and left out of most git introduction
> documentation. Relying on origin/ branches will effectively inhibit the use of
> this tool.

The question is whether the fetch factory should mess around with local branches or should always use remote branches (e.g. implicit 'origin/' prefix)? I think that it might be better to always use the remote, fetched branch.

The fetch cache is also intended as something internal to the fetch factory. It should not be touched directly by developers. Also, there is no guarantee that any local branch name will point to the same origin on any developers clone.
Comment 6 Stephan Herrmann CLA 2012-01-31 09:50:18 EST
(In reply to comment #5)
> (In reply to comment #4)
> > Local branches should be available, because the concept of origin/ is unknown
> > to most young git developers and left out of most git introduction
> > documentation. Relying on origin/ branches will effectively inhibit the use of
> > this tool.
> 
> The question is whether the fetch factory should mess around with local
> branches or should always use remote branches (e.g. implicit 'origin/' prefix)?
> I think that it might be better to always use the remote, fetched branch.

I think so, too. In my patch in bug 365944 I map no tag to "origin/master".
Just today I added (in my local workspace) similar support to expand "branches/xyz" to "origin/branches/xyz" because I was facing similar problems as in bug 369409 comment 9.

It seems we needed this prefix unconditionally :)
Comment 7 Paul Webster CLA 2012-01-31 10:00:34 EST
I like the idea of not worrying about a pull/merge to a local branch (which can put things in a weird state).

But could we just make it explicit?  Recommend as a best practice just deliberately use origin/master or origin/integration (no guess work).  And if someone occasionally uses secondaryRemote/integration for their somewhat more complex workflow, we haven't stepped in their way.

PW
Comment 8 Matthias Kempka CLA 2012-01-31 10:23:09 EST
I very much like Pauls idea, this would address my issue very well.
Comment 9 David Williams CLA 2012-08-01 22:16:02 EDT
(In reply to comment #7)
> I like the idea of not worrying about a pull/merge to a local branch (which
> can put things in a weird state).
> 
> But could we just make it explicit?  Recommend as a best practice just
> deliberately use origin/master or origin/integration (no guess work).  And
> if someone occasionally uses secondaryRemote/integration for their somewhat
> more complex workflow, we haven't stepped in their way.
> 

I'm wondering ... does this same "origin principle" apply to tags as well as branches? 

I'm asking because I'm trying to help someone who's doing maintenance, and they actually have a couple of maintenance branches, essentially in same build (I think), but, they are completely using maps with project tags to specify what to use in a build. So, I'm wondering if the same thing applies? That they should use "tag=origin/vtag" in their map entry instead of just "tag=vtag"? Especially if some of their tags come from one maintenance branch, and other tags come from another maintenance branch? 

[I don't know a lot of details ... just glanced at the issue so far ... but their problem just had the same characteristic we had in bug 377691 where the "latest" and expected changes were not being retrieved as expected.]
Comment 10 Paul Webster CLA 2012-08-10 15:47:55 EDT
(In reply to comment #9)
> 
> I'm wondering ... does this same "origin principle" apply to tags as well as
> branches? 
> 

tags aren't scoped to remotes in the same way that branches and similar references are.

If you fetch tags from 2 different remotes, they're just merged into one big pot in your git repo ... with the caveat that the same tag had better not point to different commits in the different remotes :-)

PW