Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 366279

Summary: Build should fail/abort when auto-tagging fails
Product: [Eclipse Project] Platform Reporter: John Arthorne <john.arthorne>
Component: RelengAssignee: David Williams <david_williams>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: daniel_megert, david_williams, deepakazad, kim.moir, pwebster, remy.suen, srikanth_sankaran
Version: 4.1   
Target Milestone: 4.2 RC1   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Bug Depends on:    
Bug Blocks: 375807, 376460    

Description John Arthorne CLA 2011-12-09 20:18:53 EST
When the auto-tagging script runs into a failure such as a merge conflict, the build should just abort rather than going ahead with the wrong input.
Comment 1 Dani Megert CLA 2011-12-10 00:37:33 EST
+1!
Comment 2 Kim Moir CLA 2011-12-10 08:49:23 EST
*** Bug 366282 has been marked as a duplicate of this bug. ***
Comment 3 David Williams CLA 2012-04-09 14:30:43 EDT
I think we might be close to fixing this, to the extent I understand it. What are the circumstances that would give rise to a "merge conflict"? 
Is it simply a "race" condition? If someone updates the maps after they 
have already been checked out to "scan for changes"? As things are, I think script might now fail if that happens ... though no polite messages to newsgroup about it or anything. 
 
In git-release.sh the relevant piece of code is as follows, (after getting the maps, checking for changes/updates, etc, getting ready to push back changes): 
[if not obvious, checkForErrorExit is my own function, that does a "hard exit" if return code is not 0]. 

git add $( find . -name "*.map" )
checkForErrorExit $? "Could not add maps to repository"

git commit -m "Releng build tagging for $buildTag"
# removed the following error check since, if nothing to commit, it returns 1, 
# instead of 0. 
# maybe we could use that to trigger "nothng to build" message? 
#
# where would a "merge conflict" or zombie tag fail? 
# Here? Or further down on "push"?
# checkForErrorExit $? "Could not commit to repository"

# TODO: if "nothing to commit" (no changes) then 
# the next 3 commeands would not really be needed, right? (no sense to tag, or # build, if no changes? Hence no need to push changes and push tags?

git tag -f $buildTag   #tag the map file change
checkForErrorExit $? "Could not tag repository"

git push
checkForErrorExit $? "Could not push to repository"

git push --tags
checkForErrorExit $? "Could not push tags to repository"


Education welcome.
Comment 4 David Williams CLA 2012-04-16 15:22:35 EDT
Just now, doing some "test build" (in preparation for tommorrow's I-build) 

I got message below during the auto-tagging step. Sound like what's needed? 

(mail notices, etc., will be a while ... just wanted to see if this sounds reasonable, like it is catching problems as expected). Of course, I meant to "turn off" tagging during "test builds" ... I'll do that next :/ 

git commit
[R4_HEAD 5fa4c79] Releng build auto tagging for I20120416-1508
 3 files changed, 10 insertions(+), 10 deletions(-)
git tag I20120416-1508
Updated tag 'I20120416-1508' (was 0000000)
git push
To file:///gitroot/platform/eclipse.platform.releng.maps.git
   78cc09c..5fa4c79  R4_HEAD -> R4_HEAD
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'file:///gitroot/platform/eclipse.platform.releng.maps.git'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes (e.g. 'git pull') before pushing again.  See the
'Note about fast-forwards' section of 'git push --help' for details.

   ERROR. exit code: 1  Could not push to repository
Comment 5 John Arthorne CLA 2012-04-16 17:04:55 EDT
Yes that looks good. As long as one or two people understand it I guess that's the main thing!
Comment 6 Dani Megert CLA 2012-04-17 05:44:53 EDT
(In reply to comment #5)
> Yes that looks good. As long as one or two people understand it I guess that's
> the main thing!

+1.
Comment 7 Paul Webster CLA 2012-04-17 07:36:07 EDT
(In reply to comment #4)
> Just now, doing some "test build" (in preparation for tommorrow's I-build) 
> 
> I got message below during the auto-tagging step. Sound like what's needed? 
> 
> (mail notices, etc., will be a while ... just wanted to see if this sounds
> reasonable, like it is catching problems as expected). Of course, I meant to
> "turn off" tagging during "test builds" ... I'll do that next :/ 
> 
> git commit
> [R4_HEAD 5fa4c79] Releng build auto tagging for I20120416-1508
>  3 files changed, 10 insertions(+), 10 deletions(-)
> git tag I20120416-1508
> Updated tag 'I20120416-1508' (was 0000000)
> git push
> To file:///gitroot/platform/eclipse.platform.releng.maps.git
>    78cc09c..5fa4c79  R4_HEAD -> R4_HEAD
>  ! [rejected]        master -> master (non-fast-forward)

Except from this build's point of view, this was a successful push.  R4_HEAD was the branch in question.

you get this message any time your work regularly with multiple branches in one repo.  The last pull would have updated origin/master, origin/R4_HEAD, and because R4_HEAD was checked out, R4_HEAD.  But not master.  So you work with R4_HEAD, and have no problem pushing that, but because master wasn't updated when it tries to push it it gets rejected.

From reading git push --help I believe our pushes should be:

git push origin HEAD # this pushes the current branch
or
git push origin <branch> # if we specify R4_HEAd we won't have the problem

PW
Comment 8 Paul Webster CLA 2012-04-17 10:02:53 EDT
(In reply to comment #7)
> 
> From reading git push --help I believe our pushes should be:
> 
> git push origin HEAD # this pushes the current branch
> or
> git push origin <branch> # if we specify R4_HEAd we won't have the problem

So we need to fix the auto-tagging script.  In git-release.sh:

-git push
--
+git push origin HEAD
checkForErrorExit $? "Could not push to repository"


We should try this, to only push the branch we're working on.

PW
Comment 9 David Williams CLA 2012-04-17 10:13:38 EDT
(In reply to comment #8)
> (In reply to comment #7)

> --
> +git push origin HEAD
> checkForErrorExit $? "Could not push to repository"
> 
> We should try this, to only push the branch we're working on.
> 

Ok, unless you say otherwise, I think I'll put in the explicit version, 
since we have it ... seems like a "double check" that our code is working as 
intended? 

And what about the subsequent git push --tags ... I'll assume, unless you say otherwise, that's fine the way it is? 
 

	echo "git push origin $mapVersionTag"
	git push origin $mapVersionTag
	checkForErrorExit $? "Could not push to repository"
	echo "git push tags"
	git push --tags
	checkForErrorExit $? "Could not push tags to repository"
Comment 10 Paul Webster CLA 2012-04-17 10:19:08 EDT
(In reply to comment #9)
> And what about the subsequent git push --tags ... I'll assume, unless you say
> otherwise, that's fine the way it is? 

yes, --tags is fine.

> 
> 
>     echo "git push origin $mapVersionTag"
>     git push origin $mapVersionTag

We want the branch here, that will resolve to R4_HEAD ... in my scripts, it's $relengBranch

PW
Comment 11 David Williams CLA 2012-04-17 10:27:49 EDT
(In reply to comment #10)
> (In reply to comment #9)

> > 
> >     echo "git push origin $mapVersionTag"
> >     git push origin $mapVersionTag
> 
> We want the branch here, that will resolve to R4_HEAD ... in my scripts, it's
> $relengBranch
> 
> PW

Yes, I commented in bug 376072 comment 17 that I changed $relengBranch to $mapVersionTag through out the eclipseBuilder code, 

since we seemed to use two variable names for the same thing and don't recall what it was, but that caused be to introduce a bug a few days back. 

As far as I could tell, this didn't change the way "we" interface with "your" auto tagging algorithm scripts (just changes in masterBbuild.sh and git-release.sh in eclpseBuidler). 

Let me know if I missed some hidden assumption and introduced yet another bug.
Comment 12 David Williams CLA 2012-04-17 11:17:28 EDT
For what its worth, the merge step succeeded once I "cleaned up" (i.e. deleted) the cached repo we use during the build, even without this fix, so I do think maybe I introduced something in in my "test builds" such that we ended with a "mixed" local repo cache. Either that, or it'll always work only just once. :) 

But, will still apply the fix once current I-build is complete. 

(It's signing, now).
Comment 13 David Williams CLA 2012-06-06 02:10:45 EDT
Don't recall if this was M7 or RC1, but was fixed a while back.