Bugzilla – Full Text Bug Listing
| Summary: | referenced sites are not included in Galileo Discover site | ||||||
|---|---|---|---|---|---|---|---|
| Product: | z_Archived | Reporter: | David Williams <david_williams> | ||||
| Component: | Buckminster | Assignee: | P2 Inbox <equinox.p2-inbox> | ||||
| Status: | RESOLVED FIXED | QA Contact: | |||||
| Severity: | major | ||||||
| Priority: | P3 | CC: | irbull, john.arthorne, thomas | ||||
| Version: | unspecified | ||||||
| Target Milestone: | --- | ||||||
| Hardware: | PC | ||||||
| OS: | Windows XP | ||||||
| Whiteboard: | |||||||
| Attachments: |
|
||||||
|
Description
David Williams
After some IM discussion with Pascal, and manually looking at content.xml files, I see what's missing is the <references element from the content.xml/jar file. Such as ... this is the start of one from Ganymede: <references size='58'>$ <repository url='http://download.eclipse.org/stp/updates/' type='0' options='0'/> <repository url='http://download.eclipse.org/stp/updates/' type='1' options='0'/> <repository url='http://download.eclipse.org/eclipse/updates/3.4' type='1' options='0'/> ... Thomas, I'm beginning to remember some previous conversations about this ... and something about some missing mirror API or function? Sound familiar to you? I think users will experience this as a regression. I checked our direct /webtools/milestones repo site, and it has the data in content.xml/jar, so does seem to be a function of how repo is created. See also bug 279201 for related bug. The reason this has not been fixed is that P2 doesn't really provide any way to extract the repository references, see bug 275800. I'm testing an approach now where I use Java reflection to access a private method of the LocalMetadataRepository as a temporary workaround and it looks promising. Using the new -mirrorReferences flag, the resulting site is now built with 82 referenced repositories. Personally, I must say I question the usefulness of a repository list of that size. I'll run some more tests and publish the new version shortly. *** Bug 275740 has been marked as a duplicate of this bug. *** (In reply to comment #3) > > Using the new -mirrorReferences flag, the resulting site is now built with 82 > referenced repositories. Personally, I must say I question the usefulness of a > repository list of that size. > I agree 82 is too many to be usable (since users have to find them to enable them ... and to have all enable would be a performance problem, I'm assuming). There was 58 last year, in the file. an increase of 22. Even 58 seems an oddly high number. Maybe there's some "dups" in there, such as <repository url='http://download.eclipse.org/stp/updates/' type='0' options='0'/> <repository url='http://download.eclipse.org/stp/updates/' type='1' options='0'/> I'm not sure we need two "types", what ever that is. Plus, there's 34 projects listed for this Simultaneous Release ... I'd expect there to be 34 or fewer update sites. Even that may be too many ... but how would we reduce? Ask for volunteers to be excluded? (For many, there may be no difference between their Galileo site, and their project repo ... but, for WTP there is, for example.) Ultimately, it seems a UI requirement to filter long lists ... for example, "show all", "show only repositories for installed features" and other filters users might want. (In reply to comment #5) > I'm not sure we need two "types", what ever that is. > That's one enabled and one disabled entry. I can fix so that all added repositories are disabled by default. Choosing "All available repositories" in the "Install new software" wizard will be a very slow process if we don't. > Even that may be too many ... but how would we reduce? Ask for volunteers to be > excluded? > Yes. I think you should bring this up to discussion on cross-project. I suspect that several projects may have neglected proper maintenance of their contributions to this list. (In reply to comment #5) > (In reply to comment #3) > Even 58 seems an oddly high number. Maybe there's some "dups" in there, such as > <repository url='http://download.eclipse.org/stp/updates/' type='0' > options='0'/> > <repository url='http://download.eclipse.org/stp/updates/' type='1' > options='0'/> > > I'm not sure we need two "types", what ever that is. Just as a cross-reference: types, options, etc. - see constants in IRepository * A repository type constant (value 0) representing a metadata repository. public static final int TYPE_METADATA = 0; * A repository type constant (value 1) representing an artifact repository. public static final int TYPE_ARTIFACT = 1; * An option flag constant (value 1) indicating an enabled repository. public static final int ENABLED = 1; I added some normalization so that trailing "/site.xml" and "/" are removed and all TYPE_ARTIFACT references are skipped and everything is added as DISABLED. The list shrunk from 82 to 38. Skipping the TYPE_ARTIFACT repositories shouldn't matter much since the install dialog assumes that all repositories have meta-data and artifacts as siblings in the same folder anyway. 38 references is still a high number considering how visible they are. Is that really the user experience that we want to convey with Galileo? I added a new command line option -mirrorReferences to the builder. If set, all references to meta-data repositories are mirrored into the final repository. They are all added as disabled regardless of what status they have in the source repository. The fix was released to trunk, revision 10367. I have updated hudson and I also added -mirrorReferences to the production.properties so the build that is currently executing should pick this up. I updated http://wiki.eclipse.org/Buckminster_Galileo_Builder#Command_line_options with a description of the new option. (In reply to comment #8) > > 38 references is still a high number considering how visible they are. Is that > really the user experience that we want to convey with Galileo? > I'll take a look at the sites today. If there was some relatively obvious set of 10 to 20 that should be there, would you be willing/able to include just those in the <references> ? If so, I'll take a pass at reducing them and solicit quick feedback on cross-project. Created attachment 139015 [details]
pseudo normalized list from Ganymede 58 down to 28.
So, 28 compared to 38 is an increase in 10, sounds about right ... especially since 4 tr 6 of the Ganymede sites uris look invalid.
>Skipping the TYPE_ARTIFACT repositories shouldn't matter much since the install
>dialog assumes that all repositories have meta-data and artifacts as siblings
>in the same folder anyway.
You can't omit the artifact repositories. If you only have metadata, you will be able to resolve an install operation, but then it will always fail with "artifact not found" errors when you try to complete the install. Even if the artifact repositories are co-located with the metadata, p2 won't discover them unless someone explicitly adds them. Finally, since the UI only shows metadata repositories, removing the artifact repositories won't make the list any smaller for the end user so there is no gain in removing them.
For reference, these repositories come from the "update" and "discovery" sites listed in feature.xml files. It's possible some projects did not update their feature.xml, and they are still referencing their update sites from previous releases. For example last time references were working, I recall seeing references for two different versions of CDT repositories.
(In reply to comment #12) > You can't omit the artifact repositories. If you only have metadata, you will > be able to resolve an install operation, but then it will always fail with > "artifact not found" errors when you try to complete the install > OK, in that case I misunderstood how this information was used in the install dialog. Question, does this mean that all artifacts repositories must be enabled by default? If not, how do I enable them? Reopening this to track re-adding of referenced artifact repositories. To cross-reference, I've opened bug 280090 to track the many incorrect URLs. I've "looked at" the long list in the UI (it's now available on /releases/staging) and it's not pretty, but I don't think it is so bad it should be omitted. Having it there will help to force resolution of problems, instead of letting them linger the whole release. >Question, does this mean that all artifacts repositories must be enabled by >default? If not, how do I enable them? The current end user UI doesn't expose the distinction between metadata and artifact repositories. It currently just reveals metadata repositories, but changes such as add/remove/disable/enable from the UI will operate on both the metadata and artifact repository at that location, if available. So, if you enable the metadata repo at a given location, the artifact repo will be enabled at the same time. >Having it there will help to force resolution of problems, >instead of letting them linger the whole release. The general problem of how to improve the user experience for disabled repositories is well known but we haven't found any good solution yet. One small improvement we made in 3.5 is that the "content assist" in the "Work With" field will match on disabled repositories, which makes it a bit easier to find disabled repos (for example if you type "cdt" you will find the CDT repository in the suggestion list even if it is currently disabled). Artifact repository reference reinstated. I also added two new options: -referenceIncludePattern <regexp> -referenceExcludePattern <regexp> I installed a new builder and added -referenceExcludePattern .*/site.xml to the options. The result was that the following sites were excluded: http://download.eclipse.org/tools/gef/update-site/releases/site.xml http://download.eclipse.org/tools/gef/updates/releases/site.xml http://download.eclipse.org/modeling/m2t/updates/site.xml (the last one is added anyway since it also appears without the site.xml suffix). |