Comment 1 Markus Kuppe 2010-12-04 11:35:44 EST

You will have to assign an unique port via the "org.eclipse.ecf.provider.generic.port" property per instance. The generic provider does not try to bind to a port dynamically (contrary to r-OSGi).

Comment 2 Martin Petzold 2010-12-04 12:00:08 EST

Okay, thanks:

properties.put("org.eclipse.ecf.provider.generic.port", obtainPort());

And something like this:

public static int obtainPort() throws IOException {
	ServerSocket server = new ServerSocket(0);
	int port = server.getLocalPort();
	server.close();
	return port;
}

Comment 3 Martin Petzold 2010-12-04 12:06:52 EST

I think this should be documented. At least on the ECF Provider page at ECF Generic (http://wiki.eclipse.org/ECF_Providers#Remote_Services_.28org.eclipse.ecf.remoteservice.29).

Additionaly there should be added "org.eclipse.ecf.provider.remoteservice" to Required Bundles, this is missing and not needed by other providers (right?).

Comment 4 Markus Kuppe 2010-12-04 12:29:40 EST

(In reply to comment #3)
> I think this should be documented. At least on the ECF Provider page at ECF
> Generic
> (http://wiki.eclipse.org/ECF_Providers#Remote_Services_.28org.eclipse.ecf.remoteservice.29).

Feel free to adapt the wiki to the new properties. Btw. there are more in org.eclipse.ecf.provider.generic.TCPServerSOContainer.

> Additionaly there should be added "org.eclipse.ecf.provider.remoteservice" to
> Required Bundles, this is missing and not needed by other providers (right?).

Why would you need to require a specific provider in your client code?

Comment 5 Wim Jongman 2010-12-04 16:58:39 EST

(In reply to comment #3)
> I think this should be documented. At least on the ECF Provider page at ECF
> Generic
> (http://wiki.eclipse.org/ECF_Providers#Remote_Services_.28org.eclipse.ecf.remoteservice.29).
> 
> Additionaly there should be added "org.eclipse.ecf.provider.remoteservice" to
> Required Bundles, this is missing and not needed by other providers (right?).

Hi, consider helping us document ECF.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=329124

Comment 6 Martin Petzold 2011-01-10 05:47:47 EST

For me it seems impossible with a lot of services coming and going (simultaneously) to set ports by hand. If I obtain a port and then set the property the port is still free and could be chosen by another service.

The port management should be thread safe inside of the generic provider otherwise I never get out of the JVM_Bind Exceptions.

Or is there any other suggestion?

Comment 7 Markus Kuppe 2011-01-10 06:13:34 EST

(In reply to comment #6)
> For me it seems impossible with a lot of services coming and going
> (simultaneously) to set ports by hand. If I obtain a port and then set the
> property the port is still free and could be chosen by another service.
> 
> The port management should be thread safe inside of the generic provider
> otherwise I never get out of the JVM_Bind Exceptions.
> 
> Or is there any other suggestion?

Care to provide a patch?

Markus

Comment 8 Scott Lewis 2011-01-10 10:50:30 EST

(In reply to comment #6)
> For me it seems impossible with a lot of services coming and going
> (simultaneously) to set ports by hand. If I obtain a port and then set the
> property the port is still free and could be chosen by another service.
> 
> The port management should be thread safe inside of the generic provider
> otherwise I never get out of the JVM_Bind Exceptions.
> 
> Or is there any other suggestion?

I guess I don't understand why it's necessary (or desirable) to have a single host expose multiple ECF generic server containers (which expose server ports)...that is...why not use a single ECF container, and expose multiple services via that single container?

Alternatively, you can/could control the creation of server containers/ports...by defining your own IHostContainerFinder (in 3.4) or using the upcoming Remote Service Admin implementation...bug 324215.

Comment 9 Martin Petzold 2011-01-10 11:14:10 EST

(In reply to comment #8)
> (In reply to comment #6)
> > For me it seems impossible with a lot of services coming and going
> > (simultaneously) to set ports by hand. If I obtain a port and then set the
> > property the port is still free and could be chosen by another service.
> > 
> > The port management should be thread safe inside of the generic provider
> > otherwise I never get out of the JVM_Bind Exceptions.
> > 
> > Or is there any other suggestion?
> 
> I guess I don't understand why it's necessary (or desirable) to have a single
> host expose multiple ECF generic server containers (which expose server
> ports)...that is...why not use a single ECF container, and expose multiple
> services via that single container?
> 
> Alternatively, you can/could control the creation of server
> containers/ports...by defining your own IHostContainerFinder (in 3.4) or using
> the upcoming Remote Service Admin implementation...bug 324215.

My understanding of remote services in the Specs was I just need to do the easiest to provide a remote ServiceA (set service.exported.configs and "service.exported.interfaces).

So this does not work with generic provider if I register another ServiceA, it causes a JVM_Bind Exception. As user I don't bother about how many containers there are or if multiple ports are used, I just would like to have it working on its own (like r-OSGi) and e.g. CXF-DOSGi the reference implementation.

One suggestion was to use the "org.eclipse.ecf.provider.generic.port" parameter for the generic case, but this is not manageable with multiple threads/bundles and uses many more ports as actually required(?).

I have looked at the RSA Specs just now and have to review it later, as I am not sure where the differences in basic usage of Remote Services are.

Comment 10 Scott Lewis 2011-01-10 11:31:45 EST

(In reply to comment #9)
<stuff deleted>
> My understanding of remote services in the Specs was I just need to do the
> easiest to provide a remote ServiceA (set service.exported.configs and
> "service.exported.interfaces).
> 
> So this does not work with generic provider if I register another ServiceA, it
> causes a JVM_Bind Exception. As user I don't bother about how many containers
> there are or if multiple ports are used, I just would like to have it working
> on its own (like r-OSGi) and e.g. CXF-DOSGi the reference implementation.

The spec allows configuration(provider)-specific information to be passed in service properties.  One of the ECF-specific remote service properties is one that allows a specific existing container instance to be used rather than to have a new one created with each remote service registration.  This service property is:

"org.eclipse.ecf.containerID" and is defined in 

org.eclipse.ecf.osgi.services.distribution.IDistributionConstants.SERVICE_EXPORTED_CONTAINER_ID

There's an example usage in org.eclipse.ecf.tests.osgi.services.distribution.AbstractRemoteServiceRegisterTest.testRegisterOnExistingServerWithContainerID()


> 
> One suggestion was to use the "org.eclipse.ecf.provider.generic.port" parameter
> for the generic case, but this is not manageable with multiple threads/bundles
> and uses many more ports as actually required(?).
> 
> I have looked at the RSA Specs just now and have to review it later, as I am
> not sure where the differences in basic usage of Remote Services are.

Ok.  The main difference with RSA is that RSA defines some java classes that allow the export and import process for discovery and distribution to be controlled/managed in a standard way (through standard services/classes).  The 'user interface' for remote services (i.e. the standardized remote service properties) are not changed.  

Where it's relevant to this case is that I believe you want to control the distribution process a little more than the OSGi remote service properties allow you to do...so the ECF-specific configuration and/or RSA can/could be of use.

Comment 11 Markus Kuppe 2011-01-10 12:21:55 EST

(In reply to comment #10)

> The spec allows configuration(provider)-specific information to be passed in
> service properties.  One of the ECF-specific remote service properties is one
> that allows a specific existing container instance to be used rather than to
> have a new one created with each remote service registration.  This service
> property is:
> 
> "org.eclipse.ecf.containerID" and is defined in 
> 
> org.eclipse.ecf.osgi.services.distribution.IDistributionConstants.SERVICE_EXPORTED_CONTAINER_ID
> 
> There's an example usage in
> org.eclipse.ecf.tests.osgi.services.distribution.AbstractRemoteServiceRegisterTest.testRegisterOnExistingServerWithContainerID()
> 

I don't understand why this use cases requires an ECF specific property. Shouldn't the current ECF impl of OSGi remote services just reuse an existing container in HostContainerFinder?

Comment 12 Scott Lewis 2011-01-10 12:29:51 EST

(In reply to comment #11)
> (In reply to comment #10)
> 
> > The spec allows configuration(provider)-specific information to be passed in
> > service properties.  One of the ECF-specific remote service properties is one
> > that allows a specific existing container instance to be used rather than to
> > have a new one created with each remote service registration.  This service
> > property is:
> > 
> > "org.eclipse.ecf.containerID" and is defined in 
> > 
> > org.eclipse.ecf.osgi.services.distribution.IDistributionConstants.SERVICE_EXPORTED_CONTAINER_ID
> > 
> > There's an example usage in
> > org.eclipse.ecf.tests.osgi.services.distribution.AbstractRemoteServiceRegisterTest.testRegisterOnExistingServerWithContainerID()
> > 
> 
> I don't understand why this use cases requires an ECF specific property.
> Shouldn't the current ECF impl of OSGi remote services just reuse an existing
> container in HostContainerFinder?

Yeah...in fact come to think of it it should have that behavior already (the first registration should create the container, subsequent one should find/use it).  I guess we don't understand the use case well enough...i.e. perhaps there are a number of threads registering services concurrently.

Comment 13 Martin Petzold 2011-01-10 12:34:52 EST

(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #10)
> > 
> > > The spec allows configuration(provider)-specific information to be passed in
> > > service properties.  One of the ECF-specific remote service properties is one
> > > that allows a specific existing container instance to be used rather than to
> > > have a new one created with each remote service registration.  This service
> > > property is:
> > > 
> > > "org.eclipse.ecf.containerID" and is defined in 
> > > 
> > > org.eclipse.ecf.osgi.services.distribution.IDistributionConstants.SERVICE_EXPORTED_CONTAINER_ID
> > > 
> > > There's an example usage in
> > > org.eclipse.ecf.tests.osgi.services.distribution.AbstractRemoteServiceRegisterTest.testRegisterOnExistingServerWithContainerID()
> > > 
> > 
> > I don't understand why this use cases requires an ECF specific property.
> > Shouldn't the current ECF impl of OSGi remote services just reuse an existing
> > container in HostContainerFinder?
> 
> Yeah...in fact come to think of it it should have that behavior already (the
> first registration should create the container, subsequent one should find/use
> it).  I guess we don't understand the use case well enough...i.e. perhaps there
> are a number of threads registering services concurrently.

Tthis is the use case and in fact the problem.

Comment 14 Markus Kuppe 2011-01-10 12:40:15 EST

(In reply to comment #13)

> > Yeah...in fact come to think of it it should have that behavior already (the
> > first registration should create the container, subsequent one should find/use
> > it).  I guess we don't understand the use case well enough...i.e. perhaps there
> > are a number of threads registering services concurrently.
> 
> Tthis is the use case and in fact the problem.

In that case you might wanna check org.eclipse.ecf.osgi.services.distribution.DefaultHostContainerFinder for race conditions.

Comment 15 Scott Lewis 2011-01-10 12:52:26 EST

(In reply to comment #13)
<stuff deleted>
> > it).  I guess we don't understand the use case well enough...i.e. perhaps there
> > are a number of threads registering services concurrently.
> 
> Tthis is the use case and in fact the problem.


Ok...that clarifies things.  Then for this use case, as Markus has indicated, the problem is likely the re-entrancy of the IHostContainerFinder (default implementation of DefaultHostContainerFinder class).  

Martin if you want to change the title of this bug, I would suggest something like:  

[remoteservices][osgi] host container finder should be thread safe

I will fix this problem in both the 3.4 remote services implementation and the new RSA implementation (I'm still trying to decide how to migrate from the existing 'container finder' classes to the RSA classes...which duplicate the container finder functionality, but will live in a new ECF RSA implementation bundle).

Comment 16 Martin Petzold 2011-01-10 13:05:57 EST

(In reply to comment #15)
> (In reply to comment #13)
> [...]
> Martin if you want to change the title of this bug, I would suggest something
> like:  
> 
> [remoteservices][osgi] host container finder should be thread safe
> 

I have changed the title.

> I will fix this problem in both the 3.4 remote services implementation and the
> new RSA implementation (I'm still trying to decide how to migrate from the
> existing 'container finder' classes to the RSA classes...which duplicate the
> container finder functionality, but will live in a new ECF RSA implementation
> bundle).

Just to have it clear, there are two use cases:

1) Multiple threads in one OSGi framework registering remote services
concurrently

2) Multiple OSGi framework instances (on the same machine!) registering remote
services "concurrently"

I'm not sure about the IHostContainerFinder, does the concurrent
finding/creation then cause a JVM_Bind Exception? So with the fix only one
container would be created and only one port used? So there can't be any bind
problems then anymore.

So 1) could be fixed then, just be aware of case 2) as I also had problems with
this. So bind of a port while container creation should bind directly a free
port with new ServerSocket(0) and nothing like get a free port and let time
pass by until binding.

Comment 17 Scott Lewis 2011-01-10 13:49:03 EST

Hi Martin,

(In reply to comment #16)
<stuff deleted>
> 
> Just to have it clear, there are two use cases:
> 
> 1) Multiple threads in one OSGi framework registering remote services
> concurrently

Ok...I can address this now with this bug.

> 
> 2) Multiple OSGi framework instances (on the same machine!) registering remote
> services "concurrently"

This will require some coordination/configuration for the two (or more) framework instances.  What I mean by coordination/configuration is that each framework will have to configure the host container such that it uses different ports.  This will be true whether the ECF generic or the r-OSGi provider is used, as each of these (and other providers) have a default listener port (ecf's is 3282, r-osgi's is 9278.

The configuration can/could be done via system properties (for either ecf generic or r-osgi), but I would suggest that it might be better to use the ECF-specific service property rather than a system property...as that gives you more flexibility.


> 
> I'm not sure about the IHostContainerFinder, does the concurrent
> finding/creation then cause a JVM_Bind Exception? 

The behavior of the DefaultHostContainerFinder is that *if* no matching container already exists, a new one is created (using the default port).  If multiple threads access the host container finder simultaneously, it's possible that both threads would not find an existing host container, and both then try to create one using the same port.  Based upon your description, I think this must be what's happening to you.

>So with the fix only one
> container would be created and only one port used? So there can't be any bind
> problems then anymore.

Right.

> 
> So 1) could be fixed then, just be aware of case 2) as I also had problems with
> this. So bind of a port while container creation should bind directly a free
> port with new ServerSocket(0) and nothing like get a free port and let time
> pass by until binding.

Yeah...as I stated above, for 2 there will have to be some sort of configuration that is explicitly done...so that the two or more frameworks running on a single system don't try to access the same system resource (default port).

Comment 18 Martin Petzold 2011-01-10 14:19:48 EST

(In reply to comment #17)
> Hi Martin,
> 
> (In reply to comment #16)
> <stuff deleted>
> > 
> > Just to have it clear, there are two use cases:
> > 
> > 1) Multiple threads in one OSGi framework registering remote services
> > concurrently
> 
> Ok...I can address this now with this bug.
> 
> > 
> > 2) Multiple OSGi framework instances (on the same machine!) registering remote
> > services "concurrently"
> 
> This will require some coordination/configuration for the two (or more)
> framework instances.  What I mean by coordination/configuration is that each
> framework will have to configure the host container such that it uses different
> ports.  This will be true whether the ECF generic or the r-OSGi provider is
> used, as each of these (and other providers) have a default listener port
> (ecf's is 3282, r-osgi's is 9278.
> [...]
> The behavior of the DefaultHostContainerFinder is that *if* no matching
> container already exists, a new one is created (using the default port).  If
> multiple threads access the host container finder simultaneously, it's possible
> that both threads would not find an existing host container, and both then try
> to create one using the same port.  Based upon your description, I think this
> must be what's happening to you.

I don't have this problem with r-OSGi, it is flexible "WARNING: Port 9278 already in use. This instance of R-OSGi is running on port 9279". This could also be done for generic provider (at container creation/port binding) in my opinion. It would then solve 1) and 2). So just bind a free(!) port in any case (and don't use a default). DefaultHostContainerFinder thread safe will only solve 1)!

I don't seem to have problem 1)(!) with r-OSGi, so is this really about DefaultHostContainerFinder? Yes, perhaps because r-OSGi creates one default container on bundle startup...!?

Comment 19 Scott Lewis 2011-01-10 14:37:41 EST

(In reply to comment #18)
<stuff deleted>
> > to create one using the same port.  Based upon your description, I think this
> > must be what's happening to you.
> 
> I don't have this problem with r-OSGi, it is flexible "WARNING: Port 9278
> already in use. This instance of R-OSGi is running on port 9279". This could
> also be done for generic provider (at container creation/port binding) in my
> opinion. It would then solve 1) and 2). So just bind a free(!) port in any case
> (and don't use a default). DefaultHostContainerFinder thread safe will only
> solve 1)!

That's true.

> 
> I don't seem to have problem 1)(!) with r-OSGi, so is this really about
> DefaultHostContainerFinder? Yes, perhaps because r-OSGi creates one default
> container on bundle startup...!?

There are a number of use cases where the behavior that r-osgi exhibits (of just selecting the next port...i.e. 9279) creates problems.  For example...which process gets which port is sensitive to the start order of the processes.  This can/could/will be problematic for deployment (as opposed to development). 

In my opinion, it's usually better to be explicit about port/system resource selection between processes (rather than implicitly just selecting an arbitrary open port).  This can be done easily with configuration (i.e. setting the port explicitly...e.g. r-osgi://host.com:8777 or ecftcp://host.com:9999/server or via the system properties).

But even so, you are free to suggest and hopefully contribute an enhancement to the DefaultHostContainerFinder...so that (perhaps with some option turned on/off) it exhibits this behavior of implicitly/automatically selecting an unused port for generic server container creation.  One thing to be aware of, though...with RSA the host container finder is being deprecated and replaced...with a similar service called IHostContainerSelector.  It would probably be better to focus any enhancements on the RSA IHostContainerSelector rather than IHostContainerFinder.

Comment 20 Scott Lewis 2011-01-12 17:27:32 EST

Assigning to myself.  Will address for both 3.4 host container finder as well as new rsa IHostContainerSelector.

Comment 21 Scott Lewis 2011-01-12 17:47:21 EST

Fix released to master.  Changed both DefaultHostContainerFinder class and new/RSA HostContainerSelector class.

The fix was adding the synchronized keyword to the entry point method for container finder/container selector...to prevent multiple thread from calling into the container finder/container selector simultaneously.

Resolving as fixed.

Comment 22 Martin Petzold 2011-01-15 09:59:18 EST

*** Bug 334453 has been marked as a duplicate of this bug. ***