Bugzilla – Full Text Bug Listing
| Summary: | Reconsider patch/update policy for EPP Packages | ||
|---|---|---|---|
| Product: | [Technology] EPP | Reporter: | David Williams <david_williams> |
| Component: | Packager | Assignee: | Project Inbox <epp.packager-inbox> |
| Status: | RESOLVED WONTFIX | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | P3 | CC: | beth, g.watson, irbull, jeffmcaffer, jonah, mknauer, pascal, shr31223, steffen.pingel, stephan.herrmann, t-oberlies, wayne.beaton |
| Version: | 1.3.2 | ||
| Target Milestone: | later | ||
| Hardware: | PC | ||
| OS: | Linux | ||
| See Also: | https://bugs.eclipse.org/bugs/show_bug.cgi?id=329973 | ||
| Whiteboard: | |||
|
Description
David Williams
(In reply to comment #0) > I've no idea why the WTP maintenance updates would get 'sucked in' in the above > "from scratch" scenario ... just because there are some minor, SR2-equivalent > EPP features available ... but sounds like a bug somewhere. My guess would be > "EPP meta data" but could be in p2 mechanisms, I guess? It is both, p2 and the EPP metadata. If someone is looking for updates, p2 searches for updates of the top-level feature and if there are updates available for that one, it tries to resolve all dependencies down in the hierarchy. The second fact to know is that we (usually) do not specify versions or version ranges in the feature that defines the package content. That should explain what you saw. In the mini update scenario (B) there is no update of the root feature available, therefore no updates are done at all. In the first from-scratch (A) scenario, there is an update for the top-level feature available and therefore all updates are taken into account. But I see your point that we have a problem with the SR2 update. What we could do is a rebuild of the Java EE IDE metadata (including the feature/branding plugin) and putting this on the EPP repository. > The second fact to know is that we (usually) do not specify versions or version > ranges in the feature that defines the package content. Seems we might want to do that ... for Indigo? ... if it could be automated? > > But I see your point that we have a problem with the SR2 update. > > What we could do is a rebuild of the Java EE IDE metadata (including the > feature/branding plugin) and putting this on the EPP repository. What would this rebuild look like? Do you mean specifically for Web Tools post SR2 udpate? Or just to narrow down the version ranges in SR2? I'm all for the simplest solution (or, none) for SR2? But mostly concerned if we should change our procedure for the future. Maybe I'll comment on epp-dev, to see if general agreement that "products" should be exactly defined. (In reply to comment #2) > > The second fact to know is that we (usually) do not specify versions or version > > ranges in the feature that defines the package content. > > Seems we might want to do that ... for Indigo? ... if it could be automated? > Oh, but, I just thought, if we make a product install _exact_ then that might prevent people from "manually" updating (via install) any particular component of it. That'd not be good either. Complicated. (In reply to comment #3) > > Seems we might want to do that ... for Indigo? ... if it could be automated? > > > > Oh, but, I just thought, if we make a product install _exact_ then that might > prevent people from "manually" updating (via install) any particular component > of it. That'd not be good either. Complicated. I strongly agree. A lot of projects such as Mylyn or Webtools release more frequently than Indigo (and EPP) and we generally encourage users to consume new releases. Strict version specifications would prevent users from updating off cycle. I'm wondering if a more flexible updating policy could be considered for p2 that would allow users to check for updates for non-roo IUs? Users tend to get confused that Check for Updates doesn't find anything even though they just heard about a newer Mylyn release and this causes other problems as well, e.g. bug 303260 (and I am sure it prevents a host of problems). Stepping back, from the actual mechanic, I think the discussion is about what is an EPP package. What role does it fulfill and what are the user expectations about it? Now here is my answer to this :) Given that what we are aiming to deliver as part of the EPP package are well rounded / tested set of plugins, I'm of the opinion that that packages should lock down the version of all the immediate features they include. Failing to do this means that I do not have the ability to reproduce previous versions of EPP packages, but it also means that my install is dependent on the content of the repository and the way I have obtained the download. I know that locking the features is not a 100% solution because there will always be some dependencies that are not locked through features, but locking the features included in the package would help improve the situation. As for giving the ability to the user to update to a sub piece of a package, I think this is an oversight rather than a feature, and that if new subpieces are made available, we should rebuild the packages to include them, or make patches available. But again, this is because I see EPP as a vetting / control place that delivers products and want to be sure that ppl will get what has been tested. I agree with Pascal for all the reasons he stated. This is why the "Provide p2 repository" part of the simultaneous release rules [1] we require people to use exact versions when including things in their features. Having said that, I also agree that the packages are there to server the consumers. As such we need to understand what the consumers want. Remember that people who want to slice and dice and mix and match are NOT (IMHO) the target audience here. It is the regular consumer/user. What do they expect? [1] http://www.eclipse.org/indigo/planning/EclipseSimultaneousRelease.php I'm starting to like the idea of more explicitly making "exact" versions required. Seems we might have fewer problems of people reporting odd "conflicting dependency" problems, since it would be better defined. I know its handy to think we in webtools, or mylyn, could have "off cycle releases whenever we wanted" it is not so handy to think of the lower level features "changing things" without us knowing about it. (such as if jdt, emf, or gef, ever had an offcycle release). While these low level off cycle releases might be very rare in practice, I think it demonstrates the reason for why it is an important principle. But, if not obvious, it would require some way to still allow off cycle EPP releases, or ... I like Pascal's suggestion ... using "patched EPP features" ... assuming you can have a "patch feature" for just other features ... I've only created patch features before for plugins. This is kind of a change in policy, I know the past we've sorted of insisted there only be the 3 coordinated EPP releases per year. That was, partially, to avoid "overwork" of various teams ending up wanting fixes every month, or something. But ... since some of us have off cycle releases anyway, seems it'd be nice to have the ability to have EPP updates (or, specific EPP patches) especially if could be done to minimize work to other groups. As a wild idea ... is it conceivable individual projects could release an EPP patch feature from their own repository? Instead of always having to do it from the EPP repository? While we'd certainly want to keep everyone informed, and it kind of violates normal "namespace rules" at Eclipse, it might makes thinks easier, and more logical, if, to give a concrete example, webtools wanted to do an off cycle release, they could provide a patched epp feature, if they wanted epp users to get that release, or mylyn could, if they wanted? That way, there's minimal disruption to the "common repository"? Might be clearer to users too, on what they had, or wanted, such as they might want to get the latest Mylyn patch feature, but not care about the latest web tools patch feature? > This is kind of a change in policy, I know the past we've sorted of insisted > there only be the 3 coordinated EPP releases per year. That was, partially, to > avoid "overwork" of various teams ending up wanting fixes every month, or > something. But ... since some of us have off cycle releases anyway, seems it'd > be nice to have the ability to have EPP updates (or, specific EPP patches) > especially if could be done to minimize work to other groups. If it helps you feel better, we can look at it as something precursor to the long term support plan in Eclipse :) > As a wild idea ... is it conceivable individual projects could release an EPP > patch feature from their own repository? Instead of always having to do it from > the EPP repository? While we'd certainly want to keep everyone informed, and it > kind of violates normal "namespace rules" at Eclipse, it might makes thinks > easier, and more logical, if, to give a concrete example, webtools wanted to do > an off cycle release, they could provide a patched epp feature, if they wanted > epp users to get that release, or mylyn could, if they wanted? That way, > there's minimal disruption to the "common repository"? Might be clearer to > users too, on what they had, or wanted, such as they might want to get the > latest Mylyn patch feature, but not care about the latest web tools patch > feature? A p2 patch can patch any IU anywhere in the dependency chain, which means that features can also be patched. The authoring of such a patch may not be fun (the feature.xml does not support this and neither does the p2.inf) but we have some test that show the format of such things. Also the distribution of the patch can be done from any repository. That said, rather than having a variety of patches floating around available from various sources, it seems to me that it would be easier for consumers if a full build of a given package was being done. I'll throw my hat in the ring here and say that we (PTP) would prefer if users were able to update to later releases when the become available, and not just when new EPP packages are released. We see EPP as our major delivery vehicle as we get many, many complaints about how complicated it is to get everything installed. However, we're also still undergoing vigorous development, so only having three releases a year is too restrictive. The complicating factor is that there is a "Check for updates" menu, and users find it confusing if this doesn't work as expected when we announce new releases. We could probably live if there was a single way to perform an update (e.g. via Install New Software), so one option might be to disable this menu if there are no (EPP) updates available. Otherwise more frequent EPP releases, or EPP patch features would also be ok with us, but a more flexible update policy would be our best alternative. I think it's too late to change things for Indigo (if that's not obvious). And, I suspect this is the sort of change that might be hard to do in maintenance release? But ... I wouldn't rule that out, once we have a clear policy, and learn more about how to produce "patch features" for EPP. But, the deciding factor, for me, for Indigo, is that not only a) its late with insufficient time for testing so many possible scenarios, and getting feedback from community, but, more important, b) we've been doing it this "loose" way for years already with no obviously bad effects ... since I didn't even notice :) ... and with some benefit to participating projects. So, if we change anything at all, I suggest we start early in Juno, so there'd be several milestones to evaluate and experiment with, before making final decision. I'll volunteer to write a draft of EPP update/patch policy for the EPP project ... see if we can get agreement, plenty of review ... but think the essential parts will be: * (Still) up to each package, and package maintainer's discretion, on how tightly to "tie down" the various pieces of their package. * Any participating project should be able to produce/build their own EPP Patch feature. (And, someone would need to volunteer to make that easy to do :) ... at least for a change to be feasible ... I am definitely not suggesting that Markus do more work for us! :) * Would be made available (only) from the central EPP repository and, though composition, the central release repository (at least, that is, the patch feature ... less clear to me the actual artifacts should be ... and I'm not saying not ... just need to think though still ... as I do not want even more duplication with a lot of "mini releases" ... need some sort of "patch repository", I guess). * Be well communicated to other projects in effected package, with sufficient time for comments/tests, if they desire. * Have no (or, little) impact on projects not participating in the "patch feature". (That is, not require them to change anything, test anything, etc.) As I see it, it makes a lot of sense to tie down the "lower" levels of the dependency chain ... even now, we do, apparently, tie down the lowest Eclipse Platform feature. While it might seem less risky to leave leaf components "loose", that still has issues with users getting different versions installed, depending merely on the exact time they update from the repository and which repositories they have enabled. To signify how this bugzilla entry has evolved, I've changed title, and changed it to an enhancement request since things are currently 'working as designed'. So ... we'll see. We may end up not changing anything, but I really appreciate everyone taking the time to think about this, commenting on it, and your continued help with EPP packages. Just to cross-reference, I've just now learned that for us to have "perfectly predictable installs" it would take more than just specifying feature versions with "0.0.0" and including exact versions. There are issues with "optional bundles" getting installed, since by default they are "greedy". So to some extent it depends on which repo features are installed from, and at what time (such as, if content of common repo changes, say from release to service releases, in theory we could get a different epp package). See bug 348357 and the one it it was dup'd to for more detail if interested. Just wanted to leave a trail here for general awareness, and so I would not forget. *** Bug 347511 has been marked as a duplicate of this bug. *** *** Bug 353703 has been marked as a duplicate of this bug. *** My feeling is that by looking for "the right thing to do" we're not doing justice to some unavoidable conflicts, like: - we want version lock-down in packages - some users want to use a package as a starting point, want to apply their own update strategy later - some updates should include patch features - some patch features should not be proposed as updates As an example for these conflicting requirements see bug 245299 vs. bug 350133. I don't think any one-size-fits-all solution is possible here. Instead of guessing what all users would want, why don't we allow for differentiation? I'd see two levels: For the user we could add a preference to the Automatic Updates page, like: What kinds of updates should be considered: [x] package updates [ ] feature updates [ ] patch features [ ] ... For feature providers I suggest to start from bug 329949: defining p2 metadata for describing the kind of an update. With this strategy we wouldn't have to guess what an IU *is* (an urgent fix, a new feature, something experimental?), nor to guess what a user *wants* (only stable approved updates, everything s/he can grab?). As a special bonus: if check for updates doesn't find anything propose to open the Automatic Updates preferences to select more kinds of updates. (As such my proposal is not bound to EPP, but this bug seems to be a good manifestation of one of the conflicts). The EPP project does not have its "own" Packager anymore. EPP uses other technologies, such as Eclipse Tycho, Maven and Eclipse PDE. Therefore any remaining bugs are now being closed as WONTFIX. If this bug is still relevant, please make a comment and we'll move it to the correct project/component for further investigation. This change was made as part of a bulk change. |