Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 398256

Summary: [CBI] distinguish N and I builds in repositories.txt
Product: [Eclipse Project] Platform Reporter: David Williams <david_williams>
Component: RelengAssignee: David Williams <david_williams>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P2 CC: daniel_megert, john.arthorne
Version: 4.3   
Target Milestone: 4.3 M7   
Hardware: PC   
OS: Linux   
Whiteboard:
Bug Depends on:    
Bug Blocks: 372792, 393922    

Description David Williams CLA 2013-01-15 20:27:41 EST
According to teams current work flows, some teams use "integration" to contribute to I-builds, whereas "master" should be used for N builds. 

At the moment, it is only jdt that uses this workflow, but in theory, any team could, such as if they had some "big changes" coming and might want a few weeks where nightlies were different than integration builds. 

eclipse.jdt.core.binaries: integration 
eclipse.jdt.core: integration
Comment 1 David Williams CLA 2013-01-16 10:48:24 EST
Dani, adding you to this bug, for your awareness. 

If you could live without "integration" branch (for a while) when doing an "I" build vs. and N-build is just a matter of what letter we use, and whether or not we do signing ... well, and the qualifiers, I guess. 

So, let us know if you think not having "integration" branches would be "blocker" or a future "good to have" ability.
Comment 2 Dani Megert CLA 2013-01-17 04:42:55 EST
Does this also affect things which are not part of the file or which are commented out, e.g. SWT? This is more important to me than the 'integration' branch scenario, because it allows to continue working in master during the milestone week while not contributing to re-builds (since commented out).
Comment 3 David Williams CLA 2013-01-21 02:47:12 EST
(In reply to comment #2)
> Does this also affect things which are not part of the file or which are
> commented out, e.g. SWT? This is more important to me than the 'integration'
> branch scenario, because it allows to continue working in master during the
> milestone week while not contributing to re-builds (since commented out).

With the CBI builds, the repository must stay (not be commented out) but it can be "tied down" by putting in a tag in the file instead of a branch. Then the build always gets that tag of the repository. And currently, that would be for every build N builds and I builds. So, that would allow the team to continue working in a branch ... but ... they'd not be getting builds with it. 

In theory it shouldn't take too much work to put in the smarts to say "for nightlies always take 'master'" for everything else, take what's in the repositories.txt file" (maybe half a day, to a day) ... but ... its starting to feel like I've got about 100 tasks that "shouldn't take much" to do, so just trying to prioritize. 

Thanks,
Comment 4 Dani Megert CLA 2013-01-21 02:51:57 EST
(In reply to comment #3)
> (In reply to comment #2)
> > Does this also affect things which are not part of the file or which are
> > commented out, e.g. SWT? This is more important to me than the 'integration'
> > branch scenario, because it allows to continue working in master during the
> > milestone week while not contributing to re-builds (since commented out).
> 
> With the CBI builds, the repository must stay (not be commented out) but it
> can be "tied down" by putting in a tag in the file instead of a branch. Then
> the build always gets that tag of the repository. And currently, that would
> be for every build N builds and I builds. So, that would allow the team to
> continue working in a branch ... but ... they'd not be getting builds with
> it. 
>
Not sure I fully understood now. You are saying, that I can add a tag, but not a branch? Seems strange then. I'd expect the work to make it happen for tags is about the same for a branch, if not even getting it automatically when you have on of them.
Comment 5 Dani Megert CLA 2013-01-21 02:53:40 EST
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Does this also affect things which are not part of the file or which are
> > > commented out, e.g. SWT? This is more important to me than the 'integration'
> > > branch scenario, because it allows to continue working in master during the
> > > milestone week while not contributing to re-builds (since commented out).
> > 
> > With the CBI builds, the repository must stay (not be commented out) but it
> > can be "tied down" by putting in a tag in the file instead of a branch. Then
> > the build always gets that tag of the repository. And currently, that would
> > be for every build N builds and I builds. So, that would allow the team to
> > continue working in a branch ... but ... they'd not be getting builds with
> > it. 
> >
> Not sure I fully understood now. You are saying, that I can add a tag, but
> not a branch? Seems strange then. I'd expect the work to make it happen for
> tags is about the same for a branch, if not even getting it automatically
> when you have on of them.

Or, is what you try to say: people can still use the 'integration' approach, but they will loose the N-build(s)? That would be an acceptable solution for now, until we can get the full feature back.
Comment 6 David Williams CLA 2013-01-21 03:15:45 EST
> Or, is what you try to say: people can still use the 'integration' approach,
> but they will loose the N-build(s)? That would be an acceptable solution for
> now, until we can get the full feature back.

Yep, that is exactly what I'm saying.
Comment 7 David Williams CLA 2013-03-21 18:00:06 EDT
I think everything is done, here. 

1) we don't sign on N-builds. That's been in the code for a while, and pretty sure that automatically implies jars are not conditioned and packed with pack200. 

2) Had to fix the main "maven build" call so that if BUILD_TYPE == N, then we use
-DforceContextQualifier=${BUILD_ID}" as system property to force buildId to be the qualifier. 

3) Just now added the logic to git-submodule-checkout.sh so that if BUILD_TYPE == N, then we force the branch to be master, instead of what ever branch or tag it was otherwise. And in this case we print out a message to log similar to 

INFO: Branch was forced to 'master', instead of $VAL, since doing N-build. 

Hopefully that's it.
Comment 8 David Williams CLA 2013-03-21 18:32:33 EDT
I think one "future issue" might be "what to do with Tag'd" input. 

I could imagine, as we approached the end of a release, that people _might_ want to begin work in master for the next release and not even have it built for Nightlies ... if anyone see's a great need for that kind of rule, let me know (or reopen bug). 

It is true that we can no longer simply "comment out" a line and take what ever was built before, since we don't "record" that in map files, or anything. The the aggregater itself get's a record of that commit (of what was built before), but not sure our scripts are set up to use that, if not otherwise specified in repositories.txt file. 

Paul, do you recall specifically designing for that case? 

But, otherwise, I'm content and that sort of "near release" case that's never built could easily be handled by doing the work in a side branch, until ready to have it built.
Comment 9 David Williams CLA 2013-03-21 21:36:35 EDT
One other change I forgot to mention in comment 7, we do not "tag" the repo with N-build IDs. There is no particular reason to since they are, for the most part, simply a "test build". For implementation, we still create the tags in the local version of the repo, but simply don't push them back to the origin/master.
Comment 10 David Williams CLA 2013-05-30 16:44:38 EDT
mass change to 'verified', as these bugs are either routine or obviously fixed build breaks.