Bugzilla – Full Text Bug Listing
| Summary: | Some 4.2. features and its feature branding plug-ins have lower version than in 3.8 | ||
|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Dani Megert <daniel_megert> |
| Component: | Releng | Assignee: | David Williams <david_williams> |
| Status: | VERIFIED FIXED | QA Contact: | |
| Severity: | major | ||
| Priority: | P2 | CC: | david_williams, john.arthorne, Mike_Wilson, mober.at+eclipse, overholt, pwebster |
| Version: | 4.2 | ||
| Target Milestone: | 4.2 RC4 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
|
Description
Dani Megert
Another possibility is that when the builder does qualifier substitution for these bundles/features it adds a prefix for one of the streams so they are distinct. David had another bug about this but I can't find it at the moment. Swapping the build order doesn't seem to help, because the JDT in today's 3.8 build will still appear greater than the one in yesterday's 4.2 build. On the other hand the state we're in now doesn't actually cause any problems, except maybe confusion that we have two sets of bundles with different qualifiers but the same content. If confusion is the problem then some kind of prefix on the qualifier sounds like a good answer (e.g., something like v20120514 vs v4_20120514). (In reply to comment #1) > Another possibility is that when the builder does qualifier substitution for > these bundles/features it adds a prefix for one of the streams so they are > distinct. Sounds good to me. > David had another bug about this but I can't find it at the moment. Maybe bug 378431? > On the other hand the state we're in now doesn't actually cause any problems, > except maybe confusion that we have two sets of bundles with different > qualifiers but the same content. I don't know p2 in detail, so I don't know whether it would abort to install a feature if one of its parts is older. (In reply to comment #1) > Another possibility is that when the builder does qualifier substitution for > these bundles/features it adds a prefix for one of the streams so they are > distinct. David had another bug about this but I can't find it at the moment. Bug 375654 is the one I was thinking of. (In reply to comment #2) > > On the other hand the state we're in now doesn't actually cause any problems, > > except maybe confusion that we have two sets of bundles with different > > qualifiers but the same content. > I don't know p2 in detail, so I don't know whether it would abort to install a > feature if one of its parts is older. The versions of individual bundles and features have no impact in p2 - it is all driven by the root product which lists an exact set of feature versions to install. This was a problem in Update Manager because its reconciler would always pick the newest version available. It was also a problem in Rational Installation Manager which is based on UM. It would only come up if upgrading between 3.x and 4.x though. So ... the conclusion is people want a prefix? At least for branding bundles? And ... features? ... ? I'm not sure, right off, how to allow a prefix, but suspect its the same code that is causing bug 378429. Is there a why we can just prefix one of them? Say 3.8? So we don't "set a precedent forever"? Bug 375654 is a little different, in there the concern is we end up with two bundles with same version/qualifier but subtly different code! For this bug to be "fixed right" would be kind of hard to automate. You'd basically want to alternate incrementing the qualifier "every other build" and in the other every-other to "use what the previous build used" ... which, gets very complicated when the builds are not always done in order, only only 4.2 is "rebuilt", etc. So, I wouldn't say its impossible to "fix right" ... but, more than we could do for Juno. So, if you want a prefix, you'll have to be more specific. Something like 3.8 branding bundles get prefixed with 'a' (while 4.2 remain at 'v' or "I" what ever they are now ... oh, wait, I think 'a' is greater than 'I'? I always have to check. Point is, specificity would help here. Bug 375654 could be solved by having a shared repo with 3.8 and 4.2: when a (3.8 or 4.2) build tries to build a bundle, it first checks that repo. If the bundle is already there, fetch it from there. This bug here needs another solution, which I think is a prefix. For complete clarity I would add "v38_" and "v42_". (In reply to comment #6) > This bug here needs another solution, which I think is a prefix. For complete > clarity I would add "v38_" and "v42_". If we were going to add a prefix, I'd suggest only in 3.8 and something like r38_. So even though the branding bundles in the 2 streams shouldn't interfere with each other because of p2, r38 will never be greater than the v2012 used in 4.2 PW (In reply to comment #7) > (In reply to comment #6) > > This bug here needs another solution, which I think is a prefix. For complete > > clarity I would add "v38_" and "v42_". > > If we were going to add a prefix, I'd suggest only in 3.8 and something like > r38_. So even though the branding bundles in the 2 streams shouldn't interfere > with each other because of p2, r38 will never be greater than the v2012 used in > 4.2 > > PW "r" is a no go, as we are currently using "v". (In reply to comment #8) > > "r" is a no go, as we are currently using "v". yes, but as it's being pulled in by p2, it won't matter between now and when we release 3.8. But v38 will always be higher than the v2012 we're using by default, and I wouldn't want to risk v38 (which is a temp solution until 3.8) causing problems in the future with any of our v2012 marked bundles. PW (In reply to comment #9) > (In reply to comment #8) > > > > "r" is a no go, as we are currently using "v". > > yes, but as it's being pulled in by p2, it won't matter between now and when we > release 3.8. But v38 will always be higher than the v2012 we're using by > default, and I wouldn't want to risk v38 (which is a temp solution until 3.8) > causing problems in the future with any of our v2012 marked bundles. > > PW p2 is not a good argument. If we take that, we can leave things as is as from a p2 perspective the current state won't hurt. I don't want to go from "v" to "r". I'll try to figure out where such a change might go in build scripts, in the week ahead. (In reply to comment #10) > p2 is not a good argument. If we take that, we can leave things as is as from a > p2 perspective the current state won't hurt. I don't want to go from "v" to > "r". I don't understand what is the argument for "it must start with v".. but if that is the goal then it could be something like v-3-20120514 which is less than v2012*. I'm leaning more towards the "do nothing" answer at this point though. We don't seem to have a clear picture of a problem we are trying to solve. The only potential problem I see is if the *final* build of 3.8 has a higher version than the *final* 4.2 build, it could cause problems for someone doing shared installs that pool together products based on those two versions using the legacy update reconciler. To handle this we can just make sure we do one final 4.2 build after the final 3.8 build (which could happen anyway if there is a last minute 4.2 fix). (In reply to comment #11) > I'll try to figure out where such a change might go in build scripts, in the > week ahead. I'm not optimistic, for this or bug 378429 (as I've started to look at build scripts). At least, without some changes in "autotagging" which is already over my head :) I think it used to be possible to "control" the exact tag (and its form used) because "the builder" was in charge of doing the tagging, but now, it just takes what the autotagger has already tagged. (And, to be honest, not sure how the builder used to do it ... that is, how it knew which bundles/features it was supposed to tag each build (with v${builddate} and which is was just supposed to take from the map file. ... I'm sure its in there somewhere ... just saying its not obvious to me. Adding Paul in case there's something obvious in "autotagging" that I'm not aware of ... maybe its already there, and I've just known how to use it :) (In reply to comment #14) > Adding Paul in case there's something obvious in "autotagging" that I'm not > aware of ... maybe its already there, and I've just known how to use it :) The branding features aren't auto-tagged, they're picking up the build ID. I'm hoping that's controlled by http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/tree/eclipse/buildConfigs/sdk/customTargets.xml#n172 PW (In reply to comment #12) > I'm leaning more towards the "do nothing" answer at this point though. +1 PW
> The branding features aren't auto-tagged, they're picking up the build ID. I'm
> hoping that's controlled by
> http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/tree/eclipse/buildConfigs/sdk/customTargets.xml#n172
>
You just "made my day" :)
Thanks
(In reply to comment #12) > (In reply to comment #10) > > p2 is not a good argument. If we take that, we can leave things as is as from a > > p2 perspective the current state won't hurt. I don't want to go from "v" to > > "r". > > I don't understand what is the argument for "it must start with v" Because otherwise, the version decreases when we make the change. We always want to avoid this. > The only > potential problem I see is if the *final* build of 3.8 has a higher version > than the *final* 4.2 build, Currently, we always do two I-builds: First 4.2, then 3.8, so unless we do something special, the final 3.8 build *will* have a higher version. > I'm leaning more towards the "do nothing" answer at this point though. Looks like the those who know p2 and the update manager better than me see no issue here, so I'm of course also for the simple solution at this point.
> Currently, we always do two I-builds: First 4.2, then 3.8, so unless we do
> something special, the final 3.8 build *will* have a higher version.
If it helps, we will almost always have to do a final 4.2 build, to pick up final emf contribution ... and I think its worth doing 4.2 first, in regular schedule, since then it's unit tests "get the queue" first and have a better chance of finishing in a timely manner. But agree, this will always require special attention unless/until we come up with "the right" fix, that reflects "if code doesn't change qualifier doesn't change. I know this was a special case against that rule, to purposely make the qualifier match the build timestamp, but ... I suspect that original goal needs to be "expanded" or qualified or modified in someway to deal with the "dual build" case we have today?
I wasn't around for the original discussions, but wonder if "it would be enough" to have this special case just for those (branding) bundles which really are different between the two streams:
org.eclipse.platform
org.eclipse.sdk
org.eclipse.rcp
And to no longer do it for these bundles which are supposed to be the same?
(They would then have the normal "last changed" date in version ... I guess if the "build id" in its about.ini file is slightly different and could still be done?)
org.eclipse.pde
org.eclipse.cvs
org.eclipse.help.base
org.eclipse.jdt
(In reply to comment #19) > If it helps, we will almost always have to do a final 4.2 build, to pick up > final emf contribution ... OK, didn't know that. This might just be enough as a fix for now then. > timestamp, but ... I suspect that original goal needs to be "expanded" or > qualified or modified in someway to deal with the "dual build" case we have > today? Right. At that time we did not build the same bundles two times (for two different streams). However, increasing version is always a prime directive, so one could say this got violated with how the builds are done now. > I wasn't around for the original discussions, but wonder if "it would be > enough" to have this special case just for those (branding) bundles which > really are different between the two streams: > > org.eclipse.platform > org.eclipse.sdk > org.eclipse.rcp > > And to no longer do it for these bundles which are supposed to be the same? > (They would then have the normal "last changed" date in version ... I guess if > the "build id" in its about.ini file is slightly different and could still be > done?) > > org.eclipse.pde > org.eclipse.cvs > org.eclipse.help.base > org.eclipse.jdt I don't like to use two different approaches for the same kind of bundles (i.e. branding plug-ins). (In reply to comment #20) > (In reply to comment #19) > > If it helps, we will almost always have to do a final 4.2 build, to pick up > > final emf contribution ... > > OK, didn't know that. This might just be enough as a fix for now then. > In the interest of full disclosure, what I said was true at the time I said it, but ... since, EMF has promised to do their EMF base builds Wednesday mornings (and delivered on it, today) so in an ideal week (no rebuilds required) we'd still have 4.2 come first. If many people agree its worth "resping" each RC just to correct "buildtime", please say so, else I'll assume we just need to do it once, for final build to release. And, we'll argue about (I mean, discuss :) the correct fix later. From my perspective the only interesting case is release-to-release version numbers. Between releases it doesn't matter which one gets built first. Even if you build 4.2 last, there will be a new 3.8 build next week with a greater version. Marking for RC4, to remember to do one final build for 4.2 stream, if it does not happen "naturally" as a result of respin request. I'm assuming (for now) this would be about 3 PM Eastern, on Thursday, if no 4.2 only respin requests by that point. (In reply to comment #23) > Marking for RC4, to remember to do one final build for 4.2 stream, if it does > not happen "naturally" as a result of respin request. I'm assuming (for now) > this would be about 3 PM Eastern, on Thursday, if no 4.2 only respin requests > by that point. The artificial build is 4.2-I20120607-1500. OK, the build happened, but the "trick", to simply spin another build, was too simple: since some of the bundles were not touched, they kept their old (wrong) version and hence one is higher in 3.8 than 4.2: Eclipse Help: 3.8: Version: 1.4.0.v20120528-1717-8P7vFOTFK_Qj4JmDIOYI8Tn Build id: I20120606-2100 4.2: Version: 1.4.0.v20120528-1714-8P7vFOTFK_Qj4JmDIPXM8Tn Build id: I20120607-1500 (In reply to comment #25) > OK, the build happened, but the "trick", to simply spin another build, was too > simple: since some of the bundles were not touched, they kept their old (wrong) > version and hence one is higher in 3.8 than 4.2: > > > Eclipse Help: > > 3.8: > Version: 1.4.0.v20120528-1717-8P7vFOTFK_Qj4JmDIOYI8Tn > Build id: I20120606-2100 > > 4.2: > Version: 1.4.0.v20120528-1714-8P7vFOTFK_Qj4JmDIPXM8Tn > Build id: I20120607-1500 To be explicit, this is a feature, you have found (not bundle). The branding bundle itself got incremented as expected: ./I20120606-2100/eclipse/plugins/org.eclipse.help.base_3.6.100.v201206062100.jar ./I20120607-1500/eclipse/plugins/org.eclipse.help.base_3.6.100.v201206071500.jar But, I guess the "sum" of all the help bundles don't add up enough so their "hash" is higher in 4.2 than 3.8. I wonder why all the other help bundles are "off"? (so much lower in 4.2?) Simply cherry picking order? Is there really any difference in content? If so, if there is literally new, extra content in 4.2, then I think we missed making a minor increment in the feature? (In reply to comment #26) > (In reply to comment #25) > > OK, the build happened, but the "trick", to simply spin another build, was too > > simple: since some of the bundles were not touched, they kept their old (wrong) > > version and hence one is higher in 3.8 than 4.2: > > > > > > Eclipse Help: > > > > 3.8: > > Version: 1.4.0.v20120528-1717-8P7vFOTFK_Qj4JmDIOYI8Tn > > Build id: I20120606-2100 > > > > 4.2: > > Version: 1.4.0.v20120528-1714-8P7vFOTFK_Qj4JmDIPXM8Tn > > Build id: I20120607-1500 > > To be explicit, this is a feature, you have found (not bundle). > The branding bundle itself got incremented as expected: > ./I20120606-2100/eclipse/plugins/org.eclipse.help.base_3.6.100.v201206062100.jar > ./I20120607-1500/eclipse/plugins/org.eclipse.help.base_3.6.100.v201206071500.jar > > But, I guess the "sum" of all the help bundles don't add up enough so their > "hash" is higher in 4.2 than 3.8. I wonder why all the other help bundles are > "off"? (so much lower in 4.2?) Simply cherry picking order? Is there really any > difference in content? If so, if there is literally new, extra content in 4.2, > then I think we missed making a minor increment in the feature? As you can see the change is not (only) in the hash but in the bundle version (caused by the initial problem of this bug). I think there is no difference in content, simply the icon update that had to be done in 3.8 and 4.2 because the ripo is split. But I have to check that to be sure... Touching the bundle and doing another technical build will probably fix it. > To be explicit, this is a feature, you have found (not bundle). Yep, it's the feature. > But I have to check that to be sure... I verified that they have the same content (except for the two outdated license.html files in 4.2, which are not used, because the license feature provides them. I suggest we simply delete those two files and do a 4.2 rebuild. I am ok with doing a rebuild for this. The problem is in the tag itself rather than the qualifier. If we make a change in that feature for 4.2 only it should cause it to be tagged and therefore greater. I also received a readme update for 4.2 only that I can add if there is a rebuild. (In reply to comment #29) > I am ok with doing a rebuild for this. The problem is in the tag itself rather > than the qualifier. If we make a change in that feature for 4.2 only it should > cause it to be tagged and therefore greater. > > I also received a readme update for 4.2 only that I can add if there is a > rebuild. I suggest you do that once all teams gave the go. Seems less risky than deleting some files at this point. (In reply to comment #30) > I suggest you do that once all teams gave the go. Seems less risky than > deleting some files at this point. I think we need to touch something in the help feature so its qualifier is increased in the 4.2. I was going to suggest updating the copyright in org.eclipse.help-feature/feature.properties to 2012 (in 4.2 only). Here is the help feature change to trigger retagging. Reviewed with Paul: http://git.eclipse.org/c/platform/eclipse.platform.releng.git/commit/?h=R4_HEAD&id=8f7503c3c815303ae0e88c5720ef2c42fa040471 I'll go along with what every you all decide, naturally, but could I ask you make it a little clearer what problem it is solving? ... I'm sure it is solving something, but on the surface, sounds a little-bit like its cosmetic. And not that I don't mind cosmetic fixes either ... but ... there is risk with any rebuild ... e.g. if the "signing mechanism" failed to reach TSA authority today for one bundle, then we'd have to try again tonight/tomorrow (past our deadline) [is it worth that?] ... but ... I'm sure you remember there is a risk with any rebuild. Let me know when and what you'd like. I can update community on platform-releng-dev list, if you'd like, to announce the "request" and the "time for respin" at the same time. A bit of an aside, did you notice this through systematic checking, Dani? Or just happen to notice the one with branding bundle? I'm just wondering if we have any others like this that are "slightly off" (but same contents)? Thanks, (In reply to comment #33) > I'll go along with what every you all decide, naturally, but could I ask you > make it a little clearer what problem it is solving? ... I'm sure it is solving > something, but on the surface, sounds a little-bit like its cosmetic. It is largely cosmetic. Feature versions do actually influence how p2 searches for updates. If this scenario happened with the JDT feature, you could get into problems like this: - Start with 4.2 Platform runtime - Install 4.2 JDT - Select JDT feature in Installation Details dialog and "Check for Updates" - Get prompted to upgrade to JDT from 3.8 because it has higher version In the case of help, this particular scenario is not possible because help is included in the platform feature, but a similar problem could arise if the platform was installed using the p2 reconciler which picks the highest version from the bundle pool where possible. (In reply to comment #33) > A bit of an aside, did you notice this through systematic checking, Dani? Or > just happen to notice the one with branding bundle? I'm just wondering if we > have any others like this that are "slightly off" (but same contents)? Prime directive: versions never decrease in newer builds. Besides the problems it could cause, it is also confusing for users that look at the About: what do they have to think about the fact that 4.2 includes an older version of Help? Verified that the version numbers are OK in our latest RC4 builds (3.8-I20120608-1200 and 4.2-I20120608-1400). |