Bugzilla – Full Text Bug Listing
| Summary: | Client-side discovery of repositories | ||
|---|---|---|---|
| Product: | [Eclipse Project] Equinox | Reporter: | Scott Lewis <slewis> |
| Component: | p2 | Assignee: | P2 Inbox <equinox.p2-inbox> |
| Status: | RESOLVED WONTFIX | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | P3 | CC: | bernhard.merkle, bugs.eclipse.org, contact, Ed.Merks, irbull, jeffmcaffer, Kevin_McGuire, mik.kersten, mlists, neale, pascal, ravikumar.tj, remy.suen, simon_kaegi, susan |
| Version: | 3.4 | Keywords: | helpwanted |
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Bug Depends on: | 258340 | ||
| Bug Blocks: | |||
|
Description
Scott Lewis
Two issues: the admin UI and the SDK end-user UI. - For the admin UI, people could probably start using this now by including the right ECF plug-ins (discovery view, etc.) to a workbench launch, and then the view just shows up. We'd need the discovery plug-ins in the project set, and then the question is do we want to build that view into the RCP agent version of the admin UI? That would just be an update of the RCP app perspective and whatever build stuff has to change. Scott - what would make it most useful is to provide a "copy" capability in the discovered repo view so that the users can select a URL and copy it, so they can paste that into an add repo dialog. - for the end-user UI. I could see there being some kind of "search..." button or something in the add repo dialog. So users can add files, folders, urls, or look for what's out there. I'll mark this 3.4 to keep it visible. Another thought in the spirit of "the best update UI is no UI." In situations where we can't find a required IU while building a provisioning plan. Would we ever want the planner to try to discover repos and look there? That would be sort of cool: Can't install "foo" because it requires "bar." I found "bar" at "http://groovyupdatesite.com." Do you want me to get it from there? Simon & Pascal - side question about the list of repos passed to the planner (now in the ProvisioningContext). Do the URLs in that list have to already be registered with a repo manager? Because I could just discover sites and add them to the provisioning context. Or, even cooler, the ProvisioningContext could contain advice such as: - look for discovered sites if you can't find what you need... re: Do the URLs in that list have to already be registered with a repo manager? No, (currently) these URLs are loaded on demand. e.g. if the manager is not aware of them prior to the provisioning operation since we will add them as we try to load the repository. That said, this was not my intention as the idea was to provided a mechanism to restrict the set of repositories to use. The addition of the repository in this case is incidental and I'm still trying to decide if it's cool or a liability ;) (In reply to comment #1) > Two issues: the admin UI and the SDK end-user UI. > > - For the admin UI, people could probably start using this now by including the > right ECF plug-ins (discovery view, etc.) to a workbench launch, and then the > view just shows up. We'd need the discovery plug-ins in the project set, and > then the question is do we want to build that view into the RCP agent version > of the admin UI? That would just be an update of the RCP app perspective and > whatever build stuff has to change. > > Scott - what would make it most useful is to provide a "copy" capability in the > discovered repo view so that the users can select a URL and copy it, so they > can paste that into an add repo dialog. We have an extension point for the discovery view: serviceAccessHandler. These extensions can be used to introduce context menu items for discovered services in the ECF service discovery view. When the user right-clicks on a service, if there is a serviceAccessHandler defined to recognize that service (e.g. artifact and/or meta-data repos), then an appropriate menu item can/would be added appear..."Add to Artifact Repositories" or "Add to Metadata Repositories". For example, this shows the serviceAccessHandler for update sites: http://wiki.eclipse.org/Update_Site_Discovery As you can see (down the page) it puts two menu items in the discovered update site context menu: Open Installer and add this site to remote update sites Show update site in browser We could easily add an serviceAccessHandler in p2 admin UI somewhere that recognizes artifact/metadata repo types and presents appropriate menu item as above. But this would be temporary/too much UI I expect...as see below... > > - for the end-user UI. I could see there being some kind of "search..." button > or something in the add repo dialog. So users can add files, folders, urls, or > look for what's out there. Yes, I agree. I think the ECF discovery UI is more info than people want for the Admin UI...as mentioned by Jeff in mailing list. So, it would be better to just have a new 'known repos' UI that uses the ECF discovery service directly (i.e. just asks for the previously discovered services of a certain type) and presents whatever entries exist. Then the org.eclipse.ecf.discovery.ui bundle would not be needed. I can/will point to examples of accessing the ECF discovery service directly...as this is what the discovery view does (it just always displays all information about all discovered types in a separate view). Note we've also got a discovery view enhancement request to filter the discovery view contents: #218452 (In reply to comment #3) > re: Do the URLs in that list have to already be registered with a repo manager? > > No, (currently) these URLs are loaded on demand. e.g. if the manager is not > aware of them prior to the provisioning operation since we will add them as we > try to load the repository. That said, this was not my intention as the idea > was to provided a mechanism to restrict the set of repositories to use. The > addition of the repository in this case is incidental and I'm still trying to > decide if it's cool or a liability ;) > Both, I think :) re comment 1 and the end user ui. If we go for this it might be more general to talk about it as the "known repositories" or some such. This way the knowledge of a repo is independent of the source of that knowledge. A repo may get in that list because it was discovered, because a friend/peer told the agent about it, cause the repo was in a priming list, ... So in the end user "add repo" workflow there might be a "select from known repos" and a "search for more repos" button. As for auto adding the repos to the provisioning context, that feels risky from a performance point of view. Someone is going to have to load the contents of all those repos for them to be processed and that will take time and resources and maybe start touching all manner of servers that need login etc. I think we need some underlying support in the repository managers for this to make sense. Some extension point whereby plug-ins can specify known repositories. One of these extensions would use the ECF discovery API. Another might have their own way of implementing a list of known repositories. The UI could call this API to show a list when a "look for others" button was pushed in the add site dialog. (In reply to comment #7) > I think we need some underlying support in the repository managers for this to > make sense. Some extension point whereby plug-ins can specify known > repositories. One of these extensions would use the ECF discovery API. > Another might have their own way of implementing a list of known repositories. Seems like a good idea. > > The UI could call this API to show a list when a "look for others" button was > pushed in the add site dialog. > Couldn't the extension point definition be located in a UI plugin (I don't really know the structure of the p2 UI plugins at this point, so I might be making a foolish suggestion). I'll take a look at this, no promises for Eclipse-Con but I'm trying... My intention would be to define the mechanism/extension in the main code base and then there'd be some kind of sample plugin that would use ECF discovery to do this. >Couldn't the extension point definition be located in a UI plugin (I don't
>really know the structure of the p2 UI plugins at this point, so I might be
>making a foolish suggestion).
I don't see putting it in the UI plug-in since there might be clients that are headless (or have their own alternate UI). I would imagine the extension would be defined in the same place as the metadata factory extensions. At least that is going to be my first approach...
I concur with Susan, such an extension should leave in the core. (In reply to comment #11) > I concur with Susan, such an extension should leave in the core. > OK. Just FYI...ECF's discovery API has an IDiscoveryService, which once accessed will allow either asynchronous notification of discovered services (e.g. ds.addServiceListener(listener)) or synchronous query of previously discovered services...e.g ds.getServices(). http://www.eclipse.org/ecf/org.eclipse.ecf.docs/api/org/eclipse/ecf/discovery/service/IDiscoveryService.html We have providers for discovery service based upon zeroconf/rendevous/bonjour/itunes, and based upon the Service Locator Protocol (rfc 2608). A google soc project to build a discovery provider that used heuristics to search the file system for a given service/version (e.g. eclipse itself) was built also. With all of last week's excitement, I did not get to this for M6, sorry. Moving milestone back to 3.4 Hopefully we can meet and talk some about this at the p2 BOF at EclipseCon. Hi, Scott. I'm not at EclipseCon, but of course Pascal will be at the bof. Since I won't be there, I'll just clarify what I do see happening here for this feature (nothing fancy): - extension point for supplying known p2 repos - an implementation of this extension that uses ECF discovery of the p2 repo service to supply a list of URL's - a "browse" or "search" button of some sort on the "available features" dialog that would let the user see the list of known repos The user would be relatively unaware of how the repos were discovered (whether they came from ECF discovery or some static extension that had a favorites list, or whatever.) Need to see how bug #220554 ends up before doing the ui side Per bug #220554, there can now be additional sites added to the repo managers by old update sites (and features). We don't necessarily want to populate the available software list with everything in these repos when the user didn't add them. The thinking now is that repos will have a property describing whether they are enabled (bug #226759). Automatically added repos will not be enabled. This means the UI should support a way to enable/disable these repos. In effect, we are still talking about the same UI issue. There are repos the user added, and repos we know about through some other means, and the UI needs to allow the user to include the ones we know about, either by adding them from a "known repositories" list or by enabling/disabling repos. At some point we probably need a separate bug for the different way these things are discovered (extension point, associate sites, ECF discovery, etc.) from the UI presenting them. (In reply to comment #17) <stuff deleted> > In effect, we are still talking about the same UI issue. There are repos the > user added, and repos we know about through some other means, and the UI needs > to allow the user to include the ones we know about, either by adding them from > a "known repositories" list or by enabling/disabling repos. > > At some point we probably need a separate bug for the different way these > things are discovered (extension point, associate sites, ECF discovery, etc.) > from the UI presenting them. Yes, I agree. Note that ECF discovery can be done asynchronously, meaning that ECF discovered sites could just be added to the UI-presented sites list as they are discovered...rather than requiring that the user explicitly browse. Presumably such discovered entries would be added as 'disabled' by default. A thought: would it make sense to also have UI categories for sites? (along with an enabled/disabled check box...or however rendered). Then there could be a category 'Discovered Sites' or some such. >Note that ECF discovery can be done asynchronously, meaning that ECF discovered >sites could just be added to the UI-presented sites list as they are >discovered...rather than requiring that the user explicitly browse. Presumably >such discovered entries would be added as 'disabled' by default. > >A thought: would it make sense to also have UI categories for sites? (along >with an enabled/disabled check box...or however rendered). Then there could be >a category 'Discovered Sites' or some such. I think this view would be valuable to us (implementors, admin UI users), but I'm trying to avoid a solution that requires the user to understand a "discovered site" vs. an "added site" (or an enabled site vs. a disabled site). It feels to me like we are pushing our concerns onto the user. I don't think they will care until they are looking for some particular software and don't see it. This concept has come about due to different requirements/issues. - we have a cool discovery service, let's use it, how do we want to surface it? - the old UM had associate sites, we need to record those sites and make them available when we are looking for software - due to performance issues, we don't want to look for things in discovered sites even if we've added them to the manager - we don't want to clutter the available software list with things the user doesn't know they added. So what do we think the use cases are. The ones that come to my mind are - something couldn't be added or updated because it wasn't found - they are looking for some software by name - they are curiously browsing for what software is out there For the first case, the UI could actually catch this case and look in an expanded set of repos to do a resolve if a resolve failed due to a missing dependency. For the second and third cases, we could have a "search..." or "more..." button in the available software list that would let the user browse the discovered sites (not just the sites, but their content). This way they can find exactly what they are looking for. (In reply to comment #19) <stuff deleted> > >a category 'Discovered Sites' or some such. > > I think this view would be valuable to us (implementors, admin UI users), but > I'm trying to avoid a solution that requires the user to understand a > "discovered site" vs. an "added site" (or an enabled site vs. a disabled site). > It feels to me like we are pushing our concerns onto the user. I don't think > they will care until they are looking for some particular software and don't > see it. Agreed. I think a hard part is, however, figuring out 1) where to look once they don't see it...and whether to do such looking 'automatically' 2) presenting what could be a lot of resulting information (results of search) without overwhelming them (i.e. providing some organization to resulting data). > > This concept has come about due to different requirements/issues. > - we have a cool discovery service, let's use it, how do we want to surface it? > - the old UM had associate sites, we need to record those sites and make them > available when we are looking for software > - due to performance issues, we don't want to look for things in discovered > sites even if we've added them to the manager > - we don't want to clutter the available software list with things the user > doesn't know they added. Agreed. > > So what do we think the use cases are. The ones that come to my mind are > - something couldn't be added or updated because it wasn't found > - they are looking for some software by name > - they are curiously browsing for what software is out there I agree that these are reasonable. another: - they have to install their 'normal' work environment from a corporate update site > > For the first case, the UI could actually catch this case and look in an > expanded set of repos to do a resolve if a resolve failed due to a missing > dependency. > > For the second and third cases, we could have a "search..." or "more..." button > in the available software list that would let the user browse the discovered > sites (not just the sites, but their content). This way they can find exactly > what they are looking for. > I'm not at all against having a 'search...' or 'more...'. I just think that if the use cases fit that it might be worthwhile to avoid having to search. As an example of app-level discovery, consider iTunes. When it starts up, it discovers any other shared iTunes on the LAN. All it does with this is put these discovered entries (by name) in the navigation area on left of UI (Source)...and the user can then choose to browse into those other dbs if they wish. They only present the actual songs in the remote db if the user selects the given item in navigation area. > At some point we probably need a separate bug for the different way these > things are discovered (extension point, associate sites, ECF discovery, etc.) > from the UI presenting them. So as not to hijack Scott's original bug, I opened bug #228097 for the issue of needing to show "Other" sites so that the user can choose which associate/discovery sites from update manager should be used. That is a performance issue that needs to be addressed in 3.4. This bug represents the general mechanism for adding site discovery extensions, and the UI discussion of different/cool ways we could show the discovered sites. This will be post-3.4. >For the second and third cases, we could have a "search..." or "more..." button >in the available software list that would let the user browse the discovered >sites (not just the sites, but their content). This way they can find exactly >what they are looking for. Minor issue discussed while reviewing the buttons in bug 229364. Once we add the ability to browse all site content (in the manage sites dialog or its replacement), it would make sense to remove the "Refresh" button from the available software page and move it into this view. The implementation is such that we don't know when refresh is really needed so the user just tries it when it seems like something is missing. It's not a common operation. But we can't move it to "manage sites" until the user could see content in that view. If they can't see content, they wouldn't know that they might need to refresh. Another issue, this time from bug #229364. We do not prompt with "are you sure" when a user removes a site. Also, we already have undo implemented for add/remove site. So with a new presentation of site management we can decide if we want to prompt, have undo be visible, or both. Assuming that the discovery UI will monitor repository DISCOVERED events to show what's available...see Bug 234435. At present it's possible to get multiple discovery events for the same URL. (In reply to comment #24) > Assuming that the discovery UI will monitor repository DISCOVERED events to > show what's available...see Bug 234435. At present it's possible to get > multiple discovery events for the same URL. > The following is FYI. ECF's discovery API has two ways to discover services: 1) Synchronous polling...via IDiscoveryContainerAdapter.getServices/0 and/or getServices/1. 2) Asynchronous notification...by registering listeners via IDiscoveryContainerAdapter.addServiceTypeListener, addServiceListener/1, or addServiceListener/2. The thread making the callback in 2 is arbitrary, and up to the provider...not likely to be the UI thread. Here are the javadocs for IDiscoveryContainerAdapter http://www.eclipse.org/ecf/org.eclipse.ecf.docs/api/org/eclipse/ecf/discovery/IDiscoveryContainerAdapter.html Also, we currently start/create instances of an OSGi service: IDiscoveryService which is a sub-interface of IDiscoveryContainerAdapter. So the easiest pattern for getting/using an IDiscoveryService client is: ServiceTracker discoveryTracker = new ServiceTracker(context, IDiscoveryService.class.getName(), null); discoveryTracker.open(); IDiscoveryService discoveryService = (IDiscoveryService) discoveryTracker.getService(); if (discoveryService != null) { discoveryService.addServiceListener(serviceTypeID,serviceListener); // or for synchronous access... // IServiceInfo[] serviceInfos = discoveryService.getServices(serviceTypeID); // but this can block... } For the process that is publishing the service (the service server), the relevant methods are: discoveryService.registerService(serviceInfo) and discoveryService.unregisterService(serviceInfo) Here is test code: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ecf/tests/org.eclipse.ecf.tests.discovery/?root=Technology_Project Thanks, Scott. Note that comment #7 is obsolete as far as needing to define an extension point. Now that we have defined DISCOVERY events for repositories, we don't need to define a discovery extension point for p2. >This bug represents the general mechanism for adding site discovery extensions, >and the UI discussion of different/cool ways we could show the discovered >sites. The generic mechanism is defined, so the work to be done for ECF discovery would be to use the techniques you describe and the end result is publishing DISCOVERY events. I believe that if we did this today, the manage sites dialog would automatically pick up these discovered sites. The other part of the work in this bug is revisiting the way we present repos in general. Suggestions from bug #240796: - please add a filter facility. (currently a filter is only on the main page but not in "Available Software Sites"). As the number of entries in "Available Software Sites" is usually large, it is really needed there. - maybe group urls with the same starting address into a tree ? this would make this dialog much more handy (and consistent do other p2 dialogs). At the moment it is NOT easy to find out which URL you are selection (the name tab is too small, even for the default entries, present after a usual installation) in bug 250316 we are discussing placement of the sites view (such as in the prefs page). If we did this, this would be a logical place to put prefs that control discovery. Scott, I'm starting to think about all this for M5. As I understand it, from reading http://wiki.eclipse.org/Update_Site_Discovery, the update site discovery example you did last Eclipse-Con required server-side code to (a) expose a local update site (b) register the update site service with ECF. To discover p2 sites we would need some p2 variant of (b), correct? Or is it just the same code? If I have an existing remote p2 repo, what do I have to do it to enable discovery? Have you already done this exercise? From there, things are extremely simple on the client side. We shouldn't need to worry about the ECF views or serviceAccessHandler or any of that stuff. We would just need a small bundle that uses the ECF async listeners to listen for p2 repositories, and send p2 DISCOVERY events whenever a new repo is discovered. Poof! discovered repos would show up automatically in the manage sites dialog. (In reply to comment #29) > Scott, I'm starting to think about all this for M5. As I understand it, from > reading http://wiki.eclipse.org/Update_Site_Discovery, the update site > discovery example you did last Eclipse-Con required server-side code to > (a) expose a local update site > (b) register the update site service with ECF. > > To discover p2 sites we would need some p2 variant of (b), correct? Yes, I expect it would be some variant of it...because p2 has more than just a single URL to discover (i.e. both metadata and artifact repos). Or is it > just the same code? If I have an existing remote p2 repo, what do I have to do > it to enable discovery? Have you already done this exercise? I've started it. By that I mean I started implementing a server that publish properties including...(e.g.) p2 metadata repo location (uri), artifact repo location (uri). > > From there, things are extremely simple on the client side. We shouldn't need > to worry about the ECF views or serviceAccessHandler or any of that stuff. We > would just need a small bundle that uses the ECF async listeners to listen for > p2 repositories, and send p2 DISCOVERY events whenever a new repo is > discovered. > Poof! discovered repos would show up automatically in the manage sites dialog. Yeah, I do think it will be nearly that simple for the client (good thing). The question becomes though...what discovery protocols do we want to support? The choices currently are a) Service Locator Protocol (SLP); b) zeroconf/bonjour; c) both. Markus Kuppe (our discovery API-focussed committer) is also on this bug (in Europe tz) so that he can/should advise on some of these questions about support in SLP, needed configuration/preferences, etc...as he's doing much of the lead work on the discovery API and the providers for ECF 3.0 (Galileo). (In reply to comment #30) > Yeah, I do think it will be nearly that simple for the client (good thing). > The question becomes though...what discovery protocols do we want to support? > The choices currently are > > a) Service Locator Protocol (SLP); > b) zeroconf/bonjour; > c) both. > > Markus Kuppe (our discovery API-focussed committer) is also on this bug (in > Europe tz) so that he can/should advise on some of these questions about > support in SLP, needed configuration/preferences, etc...as he's doing much of > the lead work on the discovery API and the providers for ECF 3.0 (Galileo). Which discovery provider gets used (SLP/Zeroconf/Both) is IMO a deployment issue and heavily depends on the architecture of the p2 update site server. E.g. with which privileges does the server run? Is it just another Eclipse SDK or will it be something that's supposed to be setup manually? Programmatically on the other hand, it doesn't matter which provider gets used by p2 (client/server) anyway. ECF discovery is written to simply use whatever is registered with the runtime. And here comes a question. Which ECF version is targeted by p2 (2.1 or 3.0)? If you opt for 3.0, I'd suggest to use the "OSGi way" and simply register an IServiceListener with the OSGi service registry on the client (see bug #257861). On the server side however, lookup the IDiscoveryAdvertiser via service registry and register the p2 IServiceInfos. I think it's going to be best to separate the client and server side issues into separate bugs, since different people will be involved. We will likely need a contribution from Markus/Scott on the server side to make this happen. I've opened bug 258340 for this and will continue the discussion there. Many things in the UI have changed since we were first discussing the UI for discovery, making this bug hard to follow. To help organize this bug, I am going to copy the generic suggestions for the "manage sites" dialog into a separate bug, bug 258342. I see this bug has a vote. If you are voting for ECF discovery, keep your vote here. If you are voting for one or more suggested improvements to the site UI, please change your vote to bug 258342. Sorry for all the noise. The work to be done on the client side is as described in comment #29 > From there, things are extremely simple on the client side. We shouldn't need > to worry about the ECF views or serviceAccessHandler or any of that stuff. We > would just need a small bundle that uses the ECF async listeners to listen for > p2 repositories, and send p2 DISCOVERY events whenever a new repo is > discovered. > Poof! discovered repos would show up automatically in the manage sites dialog. > and depends on decisions made on the server side (bug 258340) *** Bug 274962 has been marked as a duplicate of this bug. *** Changing title to be more accurate *** Bug 287934 has been marked as a duplicate of this bug. *** This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. This is relative to something that's not widely adopted/used. |