Community
Participate
Working Groups
This Bugzilla entry is intended to contain discussions and files related to the service autodetection for RSE. Extra information about the autodetect technology sub-group can be found in the Eclipse wiki page http://wiki.eclipse.org/index.php/DSDP/TM/Autodetect
I will be leaving this bug in NEW state until the discussions show some course of action.
Last message from the mailing list, from Felix Burton; please feel free to start a new discussions or reply it: I have played with ZeroConf outside Eclipse and have not seen any time delay, so I don't believe there is any inherit delay in the protocol. In the simplest form, ZeroConf is using multicast to communicate, so on a very busy network it might be some delay if the first multicast packet is dropped. Felix > -----Original Message----- > From: dsdp-tm-dev-bounces@eclipse.org > [mailto:dsdp-tm-dev-bounces@eclipse.org] On Behalf Of Martin Oberhuber > Sent: Friday, May 05, 2006 12:28 AM > To: Target Management developer discussions > Subject: Re: [dsdp-tm-dev] autodetect > > Hello Javier, > > I'm wondering if there is some sort of time delay until config > information gets populated to the > network with Zeroconf. When I've tested ECF discovery at > EclipseCon, I > noticed that it took > several minutes until certain services showed up. But that was just > personal impression and I > could be wrong. > > When you've converted these mailinglist discussions into > Bugzilla, could > you share the bug # > so that we can continue discussions there? [the earlier you > do it, the > less mails you'll have > to convert by copy & paste] > > thanks, > Martin
For Reference, the mailing list discussions on autodetect started with Javier's overview on dns-sd and martin's reply: http://dev.eclipse.org/mhonarc/lists/dsdp-tm-dev/msg00179.html Follow-up messages can be navigated by pressing "Thread next". For some messages, it might pay off placing individual links or copy-and paste the content here.
In the last phone conference, the issue of legacy components working with autodetect was issued. Zeroconf is fine and widely used for service discovery in a network, so I imagine that the a similar query-reply mechanism could be used over serial or USB. With this solution, legacy devices wanting to use the autodetect facility in RSE should implement a Zeroconf-like protocol over serial. Has anybody got any experience with zeroconf and serial or anybody knows if there is already a standard protocol for it ?
Hi, I have been investigating the implementation and functionality of the ECF autodiscovery using zeroconf. It looks like ECF is listening in some sort of passive mode, so only when one of the services sends a multicast packet describing itself, ECF is able to register it and carry on with the service resolution. It makes the service discovery slower than what would be desirable, as if the service is already started, it sends this information every several minutes. Possibly it could be a good issue to raise in the ECF mail list. Or possibly there is an implementation of an active zeroconf multicast query for the service "_services._dns-sd._udp.local." (asking for available services) that I couldn't find... Any idea ? > Last message from the mailing list, from Felix Burton; please feel free to > start a new discussions or reply it: > > I have played with ZeroConf outside Eclipse and have not seen any time delay, > so I don't believe there is any inherit delay in the protocol. In the simplest > form, ZeroConf is using multicast to communicate, so on a very busy network it > might be some delay if the first multicast packet is dropped. > > Felix > > > -----Original Message----- > > From: dsdp-tm-dev-bounces@eclipse.org > > [mailto:dsdp-tm-dev-bounces@eclipse.org] On Behalf Of Martin Oberhuber > > Sent: Friday, May 05, 2006 12:28 AM > > To: Target Management developer discussions > > Subject: Re: [dsdp-tm-dev] autodetect > > > > Hello Javier, > > > > I'm wondering if there is some sort of time delay until config > > information gets populated to the > > network with Zeroconf. When I've tested ECF discovery at > > EclipseCon, I > > noticed that it took > > several minutes until certain services showed up. But that was just > > personal impression and I > > could be wrong. > > > > When you've converted these mailinglist discussions into > > Bugzilla, could > > you share the bug # > > so that we can continue discussions there? [the earlier you > > do it, the > > less mails you'll have > > to convert by copy & paste] > > > > thanks, > > Martin >
I think there are two flavors of autodetect: a) discover what is available on the LAN. This is what ECF is doing by listening for "hello here I am" messages. It could probably be sped up by using dns-sd or a multicast "where are you" type message. b) knowing the IP address (or other transport) of a target, find out what services are registered there. For this type b, it appears that the target should be queried directly and no listening delay should be necessary. I'm not sure whether ECF supports such direct zeroconf queries though.
The Zeroconf protocol can be used to discover sevices independently with the link, so it looks like a good solution for legacy/attached devices, where serial is used instead of using a TCP/IP network. The protocol implementation can be the same with an abstraction layer for the communication, providing a multicast/unicast socket for TCP/IP networks or a serial interface as a COM port, USB endpoint or other custom implementations of the serial interface. This solution implies not reusing the ECF plugin, as it only provides multicast connection, and rewrite the zeroconf protocol. As zeroconf is quite an easy protocol I can't see any problem rewriting it and testing it within the RSE community. At the moment I have got an initial (basic implementation) zeroconf client prototype and the performance is much better than using the ECF plugin. As long as I finish my prototype, I'll send it to this bugzilla entry for testing. (In reply to comment #4) > In the last phone conference, the issue of legacy components working with > autodetect was issued. > Zeroconf is fine and widely used for service discovery in a network, so I > imagine that the a similar query-reply mechanism could be used over serial or > USB. > With this solution, legacy devices wanting to use the autodetect facility in > RSE should implement a Zeroconf-like protocol over serial. > Has anybody got any experience with zeroconf and serial or anybody knows if > there is already a standard protocol for it ? >
> The protocol implementation can be the same with an abstraction layer for the > communication, providing a multicast/unicast socket for TCP/IP networks or a > serial interface as a COM port, USB endpoint or other custom implementations > of the serial interface. > This solution implies not reusing the ECF plugin, as it only provides > multicast connection, and rewrite the zeroconf protocol. Could you elaborate why you think ECF is not good enough? Could ECF be extended to fill all our needs? My understanding was that ECF defines the "discovery API" which could be extractly the abstraction layer you refer to. Above this API, it should be possible to write ECF providers for Zeroconf (or other autodetect). Currently there is only TCP/IP multicast but it might make sense to write other implementations against the ECF discovery API. OSGi would load such providers as needed. Or are you creating just a prototype outside ECF now, such that it would be possible adding it to ECF later? - If we cannot use ECF, I'd be worried about creating yet another disintegrated communication mechanism. Adding Scott Lewis on CC, would you want to comment?
I haven't been on this thread so am catching up with the discussions...sorry about that. Wish I would have known sooner. (In reply to comment #8) > > The protocol implementation can be the same with an abstraction layer for the > > communication, providing a multicast/unicast socket for TCP/IP networks or a > > serial interface as a COM port, USB endpoint or other custom implementations > > of the serial interface. > > This solution implies not reusing the ECF plugin, as it only provides > > multicast connection, and rewrite the zeroconf protocol. > > Could you elaborate why you think ECF is not good enough? Could ECF be extended > to fill all our needs? > > My understanding was that ECF defines the "discovery API" which could be > extractly the abstraction layer you refer to. Above this API, it should be > possible to write ECF providers for Zeroconf (or other autodetect). This is correct. It is possible (and desireable) to have other implementations of the discovery API that either use a) some other implementation of zeroconf; b) use some *other* discovery protocol (e.g. JXTA's advertisement protocol, etc). For zeroconf it could be re-written or replaced with a new implementation...but I suspect that if the behavior desired is simply more rapid update that that could be changed simply through some parameter changes in the JmDNS code and recompile. Or maybe just a runtime parameter change. I will take a look at the parameters and see. I suspect that the JmDNS code (upon which the org.eclipse.ecf.provider.jmdns plugin is based) has some communications parameter(s) that determine freqency of update, but it's been a couple of months since I've looked at it. I'll take a look...and others are welcome to do so as well. The JmDNS source code is available here: http://sourceforge.net/projects/jmdns/ OTOH, if you want to run zeroconf on serial connection then either a new impl of zeroconf or a completely different discovery protocol could be used. In either case, I don't see a reason why the ECF discovery API could not be used. >Currently > there is only TCP/IP multicast but it might make sense to write other > implementations against the ECF discovery API. OSGi would load such providers > as needed. Correct. Further, if modifications or extensions of the discovery API itself must be made then that can also be done easily. > > Or are you creating just a prototype outside ECF now, such that it would be > possible adding it to ECF later? - If we cannot use ECF, I'd be worried about > creating yet another disintegrated communication mechanism. Adding Scott Lewis > on CC, would you want to comment? > I'm a little confused by the following statement: > This solution implies not reusing the ECF plugin, as it only provides > multicast connection, and rewrite the zeroconf protocol. The ECF core model (a IContainer for messaging) does accomodate group communication (i.e. n > 2), but it also allows for n = 2 (client/server). The same is true, I believe, of the discovery API. For example, one could (quite easily) create a discovery API implementation that was based purely on client/server messaging. In fact, this will likely be done in the future as protocols like XMPP, IAX, and JXTA have need for discovery information provided by a server (rather than via multicast).
I just read this posting carefully. (In reply to comment #5) > Hi, > > I have been investigating the implementation and functionality of the ECF > autodiscovery using zeroconf. > It looks like ECF is listening in some sort of passive mode, so only when one > of the services sends a multicast packet describing itself, ECF is able to > register it and carry on with the service resolution. > It makes the service discovery slower than what would be desirable, as if the > service is already started, it sends this information every several minutes. In tests I've done with services like itunes, I've seen discovery take on the order of 3-4 seconds. When I was at EclipseCon 2006 I left the ECF zeroconf implementation running and saw various services come in...sometimes it took 20-30 seconds for them to appear. So I believe there is a way to specify how frequently a service is published...I'll look into this for zeroconf. > Possibly it could be a good issue to raise in the ECF mail list. > Or possibly there is an implementation of an active zeroconf multicast query > for the service "_services._dns-sd._udp.local." (asking for available services) > that I couldn't find... There is the IDiscoveryContainer.getServiceInfo (synch) or IDiscoveryContainer.requestServiceInfo (asynch). Either of these should be able to do the trick. Scott
I don't know how easy/difficult is to introduce changes into the jmdns project, or if they are going to accept major changes into their code as the jmdns project is finished. To fulfill the requirements in transport in TM, we'd need to extend it to other transports than just a MulticastSocket. At the moment jmdns allows only multicasting, so serial / usb / unicast socket and other serialised transports might be necessary for TM. Possibly in the last message, when I said that ECF was not suitable, I should say that the current jmdns implementation was not suitable instead. Sorry for the confusion. (In reply to comment #9) > I haven't been on this thread so am catching up with the discussions...sorry > about that. Wish I would have known sooner. > > (In reply to comment #8) > > > The protocol implementation can be the same with an abstraction layer for the > > > communication, providing a multicast/unicast socket for TCP/IP networks or a > > > serial interface as a COM port, USB endpoint or other custom implementations > > > of the serial interface. > > > This solution implies not reusing the ECF plugin, as it only provides > > > multicast connection, and rewrite the zeroconf protocol. > > > > Could you elaborate why you think ECF is not good enough? Could ECF be extended > > to fill all our needs? > > > > My understanding was that ECF defines the "discovery API" which could be > > extractly the abstraction layer you refer to. Above this API, it should be > > possible to write ECF providers for Zeroconf (or other autodetect). > > This is correct. It is possible (and desireable) to have other implementations > of the discovery API that either use a) some other implementation of zeroconf; > b) use some *other* discovery protocol (e.g. JXTA's advertisement protocol, > etc). > > For zeroconf it could be re-written or replaced with a new implementation...but > I suspect that if the behavior desired is simply more rapid update that that > could be changed simply through some parameter changes in the JmDNS code and > recompile. Or maybe just a runtime parameter change. I will take a look at > the parameters and see. I suspect that the JmDNS code (upon which the > org.eclipse.ecf.provider.jmdns plugin is based) has some communications > parameter(s) that determine freqency of update, but it's been a couple of > months since I've looked at it. I'll take a look...and others are welcome to > do so as well. The JmDNS source code is available here: > http://sourceforge.net/projects/jmdns/ > > OTOH, if you want to run zeroconf on serial connection then either a new impl > of zeroconf or a completely different discovery protocol could be used. In > either case, I don't see a reason why the ECF discovery API could not be used. > > >Currently > > there is only TCP/IP multicast but it might make sense to write other > > implementations against the ECF discovery API. OSGi would load such providers > > as needed. > > Correct. Further, if modifications or extensions of the discovery API itself > must be made then that can also be done easily. > > > > > Or are you creating just a prototype outside ECF now, such that it would be > > possible adding it to ECF later? - If we cannot use ECF, I'd be worried about > > creating yet another disintegrated communication mechanism. Adding Scott Lewis > > on CC, would you want to comment? > > > > I'm a little confused by the following statement: > > > This solution implies not reusing the ECF plugin, as it only provides > > multicast connection, and rewrite the zeroconf protocol. > > The ECF core model (a IContainer for messaging) does accomodate group > communication (i.e. n > 2), but it also allows for n = 2 (client/server). The > same is true, I believe, of the discovery API. For example, one could (quite > easily) create a discovery API implementation that was based purely on > client/server messaging. In fact, this will likely be done in the future as > protocols like XMPP, IAX, and JXTA have need for discovery information provided > by a server (rather than via multicast). >
Hi Javier, (In reply to comment #11) > I don't know how easy/difficult is to introduce changes into the jmdns project, > or if they are going to accept major changes into their code as the jmdns > project is finished. I have access to the JmDNS 1.0 code and am in communication with Rick Blair and Arthur Van Hoff, the primary original authors. They have been quite accepting of changes/fixes I've proposed (and implemented myself). So this is possible...although I'm not sure it's the best route...see below. > To fulfill the requirements in transport in TM, we'd need to extend it to other > transports than just a MulticastSocket. At the moment jmdns allows only > multicasting, so serial / usb / unicast socket and other serialised transports > might be necessary for TM. So let me see if I understand. You want to run zeroconf protocol over serial lines? Is it really necessary that you actually run zeroconf at all? Perhaps you could just write a new ECF discovery provider...using whatever serial line discovery protocol you wish. Or you could modify jmdns...or you could look for other implementations of zeroconf to start from (and just create another ECF discovery provider plugin from those other implementations...java [preferably] or native [possible]). In any event...it's not really necessary that the jmdns provider be modified (although that is a strategy given that we have access to the source and can modify it). A new ECF discovery provider could be created using either zeroconf over serial line or some other completely different discovery protocol...whichever meets the requirements (and is doable in terms of resources). > > Possibly in the last message, when I said that ECF was not suitable, I should > say that the current jmdns implementation was not suitable instead. Sorry for > the confusion. > No problem!! Even *I* know that ECF isn't suitable for everything :)...although pushing the envelope here is my 'job'...as I'm sure you understand.
I have been thinking about how the discovered data might be displayed and persisted once obtained through DNS-SD. It could be quite useful organising it as a controller/model/view pattern, as it would give lots of advantages, some of them migh be: * Easier to mantain as the functionality gets clearly divided among the controller - model - view. In case a defect is found, it would be easier to find to which component it belongs. * Easy to extend/modify how the data, once populated in the model, is displayed. Not only a wizard can be provided, also other views or plugins wanting to use service discovery can use the model. * The controller (engine) might be improved without having to modify the model and the views. We could use a EMF based model, as it is a quite evolved and tested Eclipse project and it provides facilities not only to structure data, also to update this info and register/notify other RSE plugins if necessary.
The structure of the EMF model for the discovered services will follow the XML schema (minOccurs and maxOccurs have been removed to fit in the bug form): <?xml version="1.0" encoding="UTF-8"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.eclipse.org/rse/discovery/model" xmlns:dns-sd="http://www.eclipse.org/rse/discovery/model"> <xs:complexType name="network"> <xs:sequence> <xs:element name="device" type="dns-sd:device"/> </xs:sequence> </xs:complexType> <xs:complexType name="device"> <xs:sequence> <xs:element name="serviceType" type="dns-sd:serviceType"/> </xs:sequence> <xs:attribute name="ipAddress" type="xs:string" /> <xs:attribute name="ipName" type="xs:string" /> </xs:complexType> <xs:complexType name="serviceType"> <xs:sequence> <xs:element name="service" type="dns-sd:service"/> </xs:sequence> <xs:attribute name="name" type="xs:string" /> </xs:complexType> <xs:complexType name="service"> <xs:sequence minOccurs="1" maxOccurs="1"> <xs:element name="priority" type="xs:integer" /> <xs:element name="weight" type="xs:integer"/> <xs:element name="port" type="xs:integer"/> <xs:element name="text" type="dns-sd:text"/> </xs:sequence> <xs:attribute name="name" type="xs:string" /> </xs:complexType> <xs:complexType name="text"> <xs:attribute name="key" type="xs:string" /> <xs:attribute name="value" type="xs:string" /> </xs:complexType> </xs:schema> So results of the service discovery will be as following: <?xml version="1.0" encoding="ASCII"?> <model:network xmlns:model="http://www.eclipse.org/rse/discovery/model"> <device ipAddress="10.16.160.37" ipName="LON-JAVIERMO01.local."> <serviceType name="_ftp._tcp.local."> <service name="one._ftp._tcp.local."> <priority>0</priority> <weight>0</weight> <port>21</port> </service> <service name="two._ftp._tcp.local."> <priority>0</priority> <weight>0</weight> <port>21</port> <text key="a" value="1"/> <text key="b" value="2"/> <text key="c" value="3"/> <text key="d" value="5"/> <text key="e" value="10"/> </service> <service name="three._ftp._tcp.local."> <priority>0</priority> <weight>0</weight> <port>22</port> <text key="a1" value="1"/> <text key="b2" value="2"/> <text key="c3" value="3"/> <text key="d4" value="10"/> </service> </serviceType> </device> </model:network> Let me know if you have suggestions about the current model, as in the meantime I'll be working on the dynamic population of the model and showing the model in a view and it's quite easy to modify it. (In reply to comment #13) > I have been thinking about how the discovered data might be displayed and > persisted once obtained through DNS-SD. It could be quite useful organising it > as a controller/model/view pattern, as it would give lots of advantages, some > of them migh be: > > * Easier to mantain as the functionality gets clearly divided among the > controller - model - view. In case a defect is found, it would be easier to > find to which component it belongs. > > * Easy to extend/modify how the data, once populated in the model, is > displayed. Not only a wizard can be provided, also other views or plugins > wanting to use service discovery can use the model. > > * The controller (engine) might be improved without having to modify the model > and the views. > > We could use a EMF based model, as it is a quite evolved and tested Eclipse > project and it provides facilities not only to structure data, also to update > this info and register/notify other RSE plugins if necessary. >
Hi Javier, Few comments > > <xs:complexType name="text"> > <xs:attribute name="key" type="xs:string" /> > <xs:attribute name="value" type="xs:string" /> > </xs:complexType> dns-sd/zeroconf/bonjour/rendevour also allows byte arrays for the 'value' of the service properties...aka TXT records in dns-sd terminology. <stuff deleted> > So results of the service discovery will be as following: > > <?xml version="1.0" encoding="ASCII"?> > <model:network xmlns:model="http://www.eclipse.org/rse/discovery/model"> > <device ipAddress="10.16.160.37" ipName="LON-JAVIERMO01.local."> Is ipName needed? > <serviceType name="_ftp._tcp.local."> > <service name="one._ftp._tcp.local."> > <priority>0</priority> > <weight>0</weight> > <port>21</port> > </service> > <service name="two._ftp._tcp.local."> > <priority>0</priority> > <weight>0</weight> > <port>21</port> > <text key="a" value="1"/> > <text key="b" value="2"/> > <text key="c" value="3"/> > <text key="d" value="5"/> > <text key="e" value="10"/> > </service> > <service name="three._ftp._tcp.local."> > <priority>0</priority> > <weight>0</weight> > <port>22</port> > <text key="a1" value="1"/> > <text key="b2" value="2"/> > <text key="c3" value="3"/> > <text key="d4" value="10"/> > </service> > </serviceType> > </device> > </model:network> > > Let me know if you have suggestions about the current model, as in the meantime > I'll be working on the dynamic population of the model and showing the model in > a view and it's quite easy to modify it. ECF already has a simple view for showing/browsing discovered services. It doesn't represent the model using EMF (to avoid additional dependencies only)...but of course you can/could modify that. The view class is in org.eclipse.ecf.ui plugin, class: org.eclipse.ecf.ui.views.DiscoveryView. It's not a holy relic, but it does work. Scott
Hello Javier, just like Scott, I'm also a bit concerned about getting a dependency to the EMF runtime. This would be making the discovery plugin heavy-weight and thus harder to integrate. Given that dns-sd is an accepted standard, dynamic introspection and change notification do not seem necessary -- a static model seems to be sufficient. Looking at your proposed model this would just be 3 interfaces and thus simple to write: IDiscoveredDevice, IDiscoveredServiceType, IDiscoveredService. What unique advantages of using EMF do you see as opposed to such hand-written interfaces? - As far as I know, EMF should be able to read a model out of such interfaces if we want to migrate to EMF later. Regarding the model, I was surprised that you were using <device ipAddress="10.16.160.37" ipName="LON-JAVIERMO01.local."> to identify the device. In one of your previous posts you had said that you were interested in using the discovery service over USB and serial lines as well. How would such devices be identified? Would a "uniqueName" property or "rseConnectionName" property perhaps be more useful? Finally, I'm afraid I'm not yet seeing the big picture of what your service will eventually look like. What use cases (for the Symbian devices e.g.) do you want to cover? What should the user experience for autodetect eventually be? Thanks a lot for your work.
Created attachment 43255 [details] Service Discovery Wizard Use Case
Hi, > just like Scott, I'm also a bit concerned about getting a dependency to the EMF > runtime. This would be making the discovery plugin heavy-weight and thus harder > to integrate. Given that dns-sd is an accepted standard, dynamic introspection > and change notification do not seem necessary -- a static model seems to be > sufficient. Looking at your proposed model this would just be 3 interfaces and > thus simple to write: > IDiscoveredDevice, IDiscoveredServiceType, IDiscoveredService. > What unique advantages of using EMF do you see as opposed to such hand-written > interfaces? - As far as I know, EMF should be able to read a model out of such > interfaces if we want to migrate to EMF later. After investigating possible models, I found EMF the best solution, as it was widely used and well maintained Eclipse project. It not only generates the model for you, it also (and most important) provides change notification and printing facilities. I think going for a standard modelling framework is much better than reinventing the weel and having to create everything from scratch. So it gives more time to concentrate on the engine and not having to be bothered by reimplementing a UI notification system, for example. > Regarding the model, I was surprised that you were using > <device ipAddress="10.16.160.37" ipName="LON-JAVIERMO01.local."> > to identify the device. In one of your previous posts you had said that you > were interested in using the discovery service over USB and serial lines as > well. How would such devices be identified? Would a "uniqueName" property or > "rseConnectionName" property perhaps be more useful? I changed ipAddress to be address and ipName to be name, as it is more general. Address will be a string identifying the device, the IP for TCPIP based communications seems to be the best, but for serial, USB etc. it may be "COM1" or any other identifier. The attribute "name" is not going to be always available, in zeroconf it is provided as part as the response from the device if it has a name allocated by a DNS server. Please let me know if you have more proposals/questions about the model. > Finally, I'm afraid I'm not yet seeing the big picture of what your service > will eventually look like. What use cases (for the Symbian devices e.g.) do you > want to cover? What should the user experience for autodetect eventually be? I sent a sequence diagram of the Service Discovery wizard use case to the mailing list some weeks ago (26/04/06), I'll attach it in the bugzilla entry and it would be great if you could add it to the wiki page. This way it'll be more accessible and then easier to understand what service discovery is going to provide. I'm working on an updated version of the use case, but the current one might give an idea about the service discovery usage. Cheers, Javier (In reply to comment #16) > Hello Javier, > > just like Scott, I'm also a bit concerned about getting a dependency to the EMF > runtime. This would be making the discovery plugin heavy-weight and thus harder > to integrate. Given that dns-sd is an accepted standard, dynamic introspection > and change notification do not seem necessary -- a static model seems to be > sufficient. Looking at your proposed model this would just be 3 interfaces and > thus simple to write: > IDiscoveredDevice, IDiscoveredServiceType, IDiscoveredService. > What unique advantages of using EMF do you see as opposed to such hand-written > interfaces? - As far as I know, EMF should be able to read a model out of such > interfaces if we want to migrate to EMF later. > > Regarding the model, I was surprised that you were using > <device ipAddress="10.16.160.37" ipName="LON-JAVIERMO01.local."> > to identify the device. In one of your previous posts you had said that you > were interested in using the discovery service over USB and serial lines as > well. How would such devices be identified? Would a "uniqueName" property or > "rseConnectionName" property perhaps be more useful? > > Finally, I'm afraid I'm not yet seeing the big picture of what your service > will eventually look like. What use cases (for the Symbian devices e.g.) do you > want to cover? What should the user experience for autodetect eventually be? > > Thanks a lot for your work. >
Hi Javier, (In reply to comment #18) <stuff deleted> > > thus simple to write: > > IDiscoveredDevice, IDiscoveredServiceType, IDiscoveredService. Or, in the case of ECF discovery interfaces: IServiceEvent (asynch service resolution notificaiton), and IServiceInfo (info about a service). > > What unique advantages of using EMF do you see as opposed to such hand-written > > interfaces? - As far as I know, EMF should be able to read a model out of such > > interfaces if we want to migrate to EMF later. > > After investigating possible models, I found EMF the best solution, as it was > widely used and well maintained Eclipse project. It not only generates the > model for you, it also (and most important) provides change notification and > printing facilities. I agree completely that EMF is a great framework for representing these sorts of data structures, but it seems like perhaps a little overkill for services info (i.e. from zeroconf or anything other discovery protocol). And it does come at the cost of requiring EMF plugins...not a problem at all for full Eclipse installations, of course, but potentially a problem for devices that are running OSGI runtime + a few plugins (I'm not sure whether that's a target runtime or not for DSDP). But I do agree that for a reasonably complete wizard, with db of services that EMF is a perfect choice. > I think going for a standard modelling framework is much better than > reinventing the weel and having to create everything from scratch. So it gives > more time to concentrate on the engine and not having to be bothered by > reimplementing a UI notification system, for example. I agree, but are those things (e.g. UI notification system) needed? I suppose I just don't know what's being planned here in terms of discovery client functionality. > > > Regarding the model, I was surprised that you were using > > <device ipAddress="10.16.160.37" ipName="LON-JAVIERMO01.local."> > > to identify the device. In one of your previous posts you had said that you > > were interested in using the discovery service over USB and serial lines as > > well. How would such devices be identified? Would a "uniqueName" property or > > "rseConnectionName" property perhaps be more useful? > > I changed ipAddress to be address and ipName to be name, as it is more general. > Address will be a string identifying the device, the IP for TCPIP based > communications seems to be the best, but for serial, USB etc. it may be "COM1" > or any other identifier. The attribute "name" is not going to be always > available, in zeroconf it is provided as part as the response from the device > if it has a name allocated by a DNS server. > Please let me know if you have more proposals/questions about the model. With the IP address (i.e. InetAddress) coming in response to the zeroconf service resolution couldn't the client just lookup the ipName (given a DNS server is available)? Given that the ipName is optional in dns-sd responses wouldn't it make more sense to do that. > > > Finally, I'm afraid I'm not yet seeing the big picture of what your service > > will eventually look like. What use cases (for the Symbian devices e.g.) do you > > want to cover? What should the user experience for autodetect eventually be? > > I sent a sequence diagram of the Service Discovery wizard use case to the > mailing list some weeks ago (26/04/06), I'll attach it in the bugzilla entry > and it would be great if you could add it to the wiki page. This way it'll be > more accessible and then easier to understand what service discovery is going > to provide. I'm working on an updated version of the use case, but the current > one might give an idea about the service discovery usage. Cool. Understanding what you have in mind (and what target environments it's meant/required to run on) is probably the critical bit here for determining need WRT EMF, etc. Scott
Created attachment 44644 [details] Service Discovery plugin beta
(In reply to comment #20) > Created an attachment (id=44644) [edit] > Service Discovery plugin beta > I'm still a little puzzled as to why it's necessary (or desireable) to define a new model to represent (e.g.) service info, service type identifiers, etc specifically for RSE when the ECF discovery API provides an open equivalent model (i.e. org.eclipse.ecf.discovery). Note that using the ECF interfaces doesn't preclude using EMF to create an ecore model, generate viewers and editors, etc. Note also that if changes/additions to the API itself need to be made then that can/could be accomodated as well. (As well as using alternative implementations of zeroconf or other discovery protocols). It still seems to me as if the discovery model...as distinct from any specific protocol implementation/provider...makes sense as something to be defined and shared across Eclipse projects. That is, how good will it be if DSDP, platform, web tools, etc., etc all have their own discovery mechanisms...both APIs and implementations...all of which could be incompatible and perhaps even conflicting. I know that DSDP has 'worry about it's own' needs, but it seems to me as if this is a perfect opportunity to not have all the projects balkanize and reinvent/reimplement the same thing over and over. Seems to me that's something worth avoiding.
I also think it would be a good idea to have a common model for service discovery. The architecture of the service discovery engine for RSE doesn't prevent populating other models (as the ECF one or others) but I just implemented the EMF one, mainly due to time constraints. As I said before, I'm using EMF because is supposed to be the standard modelling tool for Eclipse community, it's quite accepted, documented, maintained and easily understendable by the average developer. As the code is open source, I'm expecting this code to be extended to fulfill the community requirements, not only with other transport implementation (TCP/IP, serial, USB...), also populating other models if required. (In reply to comment #21) > (In reply to comment #20) > > Created an attachment (id=44644) [edit] > > Service Discovery plugin beta > > > > I'm still a little puzzled as to why it's necessary (or desireable) to define a > new model to represent (e.g.) service info, service type identifiers, etc > specifically for RSE when the ECF discovery API provides an open equivalent > model (i.e. org.eclipse.ecf.discovery). > > Note that using the ECF interfaces doesn't preclude using EMF to create an > ecore model, generate viewers and editors, etc. Note also that if > changes/additions to the API itself need to be made then that can/could be > accomodated as well. (As well as using alternative implementations of zeroconf > or other discovery protocols). > > It still seems to me as if the discovery model...as distinct from any specific > protocol implementation/provider...makes sense as something to be defined and > shared across Eclipse projects. That is, how good will it be if DSDP, > platform, web tools, etc., etc all have their own discovery mechanisms...both > APIs and implementations...all of which could be incompatible and perhaps even > conflicting. I know that DSDP has 'worry about it's own' needs, but it seems > to me as if this is a perfect opportunity to not have all the projects > balkanize and reinvent/reimplement the same thing over and over. Seems to me > that's something worth avoiding. >
I think the point is not so much that your implementation could be an ECF provider and thus fill an ECF discovery model; the point is more the other way round -- if somebody registers an ECF discovery provider that works completely different but fulfills the ECF interfaces, we'd probably want this to be available to the RSE as well and make use of it immediately. I could imagine that such ECF integration could perhaps be added later, but please keep in mind that an important point of the Eclipse Community is working together and leveraging each other's achievements (and not re-doing the same thing in different ways). Regarding EMF, I'm also worried a little bit about depending on such a large package if not necessary. But I do understand your time constraints, and it looks like the model is simple enough to allow getting rid of EMF at a later time if that is desired -- As we have done with the RSE model.
The upcoming Eclipse Technology / Aperi project is going to work on Autodetect of storage devices through SMI-S, which is based on the SLPv2 protocol (Service Location Protocol v2 -RFC2608-). It looks like it provides quite the same functionality than DNS-SD, but using customised queries. It seems important that the TM Autodetect implementation supports plugging in other discovery protocols, or even discovery services that are not network based at all (e.g. by inspecting a Kernel image). Plugging in alternative discovery methods might be possible by a factory in the proposed plugins, and/or by using the ECF discovery API with OSGi services used by ECF.
Created attachment 45747 [details] RSE Service Discovery beta screening Find attached a screening of the current version of service discovery for RSE. I think it can be really useful to check if it fulfills the requirements for service discovery, and discuss and change anything if necessary. Realise that due to the constraints on the RSE model (it is system oriented rather than service oriented) a new model entry (top level tree brach) has to be created for each discovered service.
(In reply to comment #25) Thanks for the video. It seems that you've made excellent progress! - Here are some comments. Your example shows that as soon as "Service Discovery" is enabled on the first wizard page, the settings for "host name" and "system type" get disabled and are no longer useful. I think that's exacly the sort of use case that is supported by the new "newConnectionWizardDelegate" extension point that Kushal has added for RSE M3. You should consider porting your code to this new M3 feature! Then, adding a new connection of type "Discovery" could initially show the tree view page which makes more sense. On your second wizard page, the tree shows several discovered services for which no RSE subsystems/services are installed (http,telnet,stat). I think that these should be hidden or disabled, because selecting them makes no sense. If the user still wants to see those, there could be a checkbox "Show unsupported services" Finally - you have selected ssh.shells and ftp.files on the same host. I cannot quite see why there need to be multiple root nodes for these? - The wizard should be able to collapse several services that were selected on the same host into a single node.
Created attachment 45911 [details] Service Discovery plugin Beta v2 Updated packaging of the service discovery beta. The source code has not been updated.
> Your example shows that as soon as "Service Discovery" is enabled on the first > wizard page, the settings for "host name" and "system type" get disabled and > are no longer useful. I think that's exacly the sort of use case that is > supported by the new "newConnectionWizardDelegate" extension point that Kushal > has added for RSE M3. You should consider porting your code to this new M3 > feature! Then, adding a new connection of type "Discovery" could initially show > the tree view page which makes more sense. I'm investigating at the moment how to adapt the code to the RSE M3 and use the newConnectionWizardDelegate. Please let me know if you have any suggestions or guidelines about how it might be done. > > On your second wizard page, the tree shows several discovered services for > which no RSE subsystems/services are installed (http,telnet,stat). I think that > these should be hidden or disabled, because selecting them makes no sense. If > the user still wants to see those, there could be a checkbox "Show unsupported > services" I'll investigate how to implement it. It looks like it can be done by filtering the displayed services on the tree or graying the non-supported ones. > > Finally - you have selected ssh.shells and ftp.files on the same host. I cannot > quite see why there need to be multiple root nodes for these? - The wizard > should be able to collapse several services that were selected on the same host > into a single node. This is quite tricky, as I said before the RSE model just supports creating hosts of a specific systemType. Since service discovery cannot be linked to a static list of services (they will be discovered at runtime) being able to group different services together implies changing how a new host is created. I think changing this behaviour makes sense so new hosts added through service discovery would be consistent with the current host tree display. If nobody disagrees I think it may be useful opening a bugzilla defect requiring being able create new hosts with independent services (not the static ones, as at the moment) and starting a discussion about how it could be done.
(In reply to comment #27) > Created an attachment (id=45911) [edit] > Service Discovery beta v2 Javier -- In the .classpath files, the source entry is missing: <classpathentry kind="src" path="src"/> I'm wondering how you created these projects, and how you are able to build them?
Created attachment 46888 [details] Service Discovery plugin beta for RSE1.0 M3 Service Discovery plugin tested with RSE 1.0 M3. To use it, it's necesseary to patch the standard RSE (with the attached patches) to create services as a workaround for bug #150265
Created attachment 46889 [details] RSE 1.0 M3 patches for service discovery Patches to create a "Discovery" systemType and adapt ftp and ssh to match the standard service names. This is intended to be a workaround until bug #150265 is solved
Created attachment 46936 [details] Service Discovery screen record for RSE 1.0 M3 Screen record of the usage of service discovery
(In reply to comment #32) > Created an attachment (id=46936) [edit] > Service Discovery screen record for RSE 1.0 M3 > > Screen record of the usage of service discovery > Hi Javier, I would like to use your implementation to create an ECF discovery provider (i.e. put the ECF discovery API around your implementation)...in answer to Martin's suggestion from comment #24: >>Plugging in alternative discovery methods might be possible by a factory in >>the >>proposed plugins, and/or by using the ECF discovery API with OSGi services >>used >>by ECF. Is the source code for these plugins checked into dev.eclipse.org somewhere? If so, where? If not is there a latest copy of things that I could get? Work looks very cool, BTW. Thanksinadvance, Scott
Hi Scott, The service discovery engine can be downloaded from https://bugs.eclipse.org/bugs/attachment.cgi?id=46888 (attached to this entry) The code has still to be reviewed before being added to the Eclipse repository. The service discovery engine is RSE independent (org.eclispe.tm.discovery pakages) and contains a view to start the service discovery wizard. Please, don't hesitate to contact me for any question about the code or the architecture of the plugins. Javier (In reply to comment #33) > (In reply to comment #32) > > Created an attachment (id=46936) [edit] > > Service Discovery screen record for RSE 1.0 M3 > > > > Screen record of the usage of service discovery > > > > Hi Javier, > > I would like to use your implementation to create an ECF discovery provider > (i.e. put the ECF discovery API around your implementation)...in answer to > Martin's suggestion from comment #24: > > >>Plugging in alternative discovery methods might be possible by a factory in >>the > >>proposed plugins, and/or by using the ECF discovery API with OSGi services >>used > >>by ECF. > > Is the source code for these plugins checked into dev.eclipse.org somewhere? > If so, where? If not is there a latest copy of things that I could get? > > Work looks very cool, BTW. Thanksinadvance, > > Scott >
Hi Javier, I'll retrieve, thanks. I assume the plan for licensing, etc *is* to put this code under EPL and commit to the eclipse.org repository under the dsdp project...is that correct? And I'll take a look at the plugins to figure this out...but I assume that you've factored out the UI code, the EMF model stuff, and the zeroconf impl into separate plugins? Also...are there any dependencies on any other third party libs? Thanks, Scott (In reply to comment #34) > Hi Scott, > > The service discovery engine can be downloaded from > https://bugs.eclipse.org/bugs/attachment.cgi?id=46888 (attached to this entry) > The code has still to be reviewed before being added to the Eclipse repository. > > The service discovery engine is RSE independent (org.eclispe.tm.discovery > pakages) and contains a view to start the service discovery wizard. > > Please, don't hesitate to contact me for any question about the code or the > architecture of the plugins. > > Javier > > > (In reply to comment #33) > > (In reply to comment #32) > > > Created an attachment (id=46936) [edit] > > > Service Discovery screen record for RSE 1.0 M3 > > > > > > Screen record of the usage of service discovery > > > > > > > Hi Javier, > > > > I would like to use your implementation to create an ECF discovery provider > > (i.e. put the ECF discovery API around your implementation)...in answer to > > Martin's suggestion from comment #24: > > > > >>Plugging in alternative discovery methods might be possible by a factory in >>the > > >>proposed plugins, and/or by using the ECF discovery API with OSGi services >>used > > >>by ECF. > > > > Is the source code for these plugins checked into dev.eclipse.org somewhere? > > If so, where? If not is there a latest copy of things that I could get? > > > > Work looks very cool, BTW. Thanksinadvance, > > > > Scott > > >
In order to make easier adopting service discovery to the community, I would suggest discovering system types instead of subsystem id's, as it may be easier to adopt. At the moment we have FTP and SSH, that are subsystems, but also mapped as system types, so I would propose linking the remotely advertised services with the available system types. It would work for FTP and SSH as simple services, but also would allow advertising dstore servers. So, with the current RSE, we would be able to discover the following services through DNS-SD : * Daytime * FTP * Linux * SSH * Unix * Windows
I would like to suggest that DSDP consider using the ECF discovery API (along with Javier's zeroconf implementation). After recent API refactoring (0.9.3) , ECF's core plugins are now down to code size of 30K (identity), + 50K (core) + 10K (discovery) for needed plugins. These plugins now work on the CDC 1.0/Foundation 1.0 execution environments, making it suitable for use in restricted/small runtimes. The ECF discovery API also supports discovery by service type and by name. It's also extensible via adapters if necessary. With a very little effort Javier's discovery implementation can/could be made into ECF provider. Finally, ECF now has an asynchronous file transfer API (org.eclipse.ecf.filetransfer) and are working on implementations of this API based upon apache httpclient 3.0.1, and bittorrent and have one based upon JRE URLConnection. We can/could easily use other file transfer providers (e.g. (In reply to comment #36) > In order to make easier adopting service discovery to the community, I wo0uld > suggest discovering system types instead of subsystem id's, as it may be easier > to adopt. > > At the moment we have FTP and SSH, that are subsystems, but also mapped as > system types, so I would propose linking the remotely advertised services with > the available system types. > > It would work for FTP and SSH as simple services, but also would allow > advertising dstore servers. So, with the current RSE, we would be able to > discover the following services through DNS-SD : > > * Daytime > * FTP > * Linux > * SSH > * Unix > * Windows >
I'm closing this bug since the TM Discovery component is being retired as of TM 3.3M4. TCF is the means of choice for autodiscovery in the embedded space. For protocols other than TCF, use Eclipse ECF.
