Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 427363

Summary: m2m.eclipse.org is very slow
Product: Community Reporter: Ian Skerrett <ian.skerrett>
Component: vserversAssignee: Eclipse Webmaster <webmaster>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: contact, denis.roy, roger
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows 7   
Whiteboard:

Description Ian Skerrett CLA 2014-02-04 09:12:05 EST
We are setting up a redirect to iot.eclipse.org but now the web site for m2m.eclipse.org is very slow and often times out.  Can webmaster please take a look.
Comment 1 Benjamin Cabé CLA 2014-02-04 09:22:51 EST
Denis if you send me your public key I'd be happy to give you root access to the vserver.
Comment 2 Denis Roy CLA 2014-02-05 13:30:00 EST
There were 998 connections (out of 1000) to incoming port 1883.  Is that expected?  I've cleared all incoming connections, and the site is responsive now.

I can increase the max. connections to the host, but I want to make sure this behaviour is expected.
Comment 3 Benjamin Cabé CLA 2014-02-05 13:36:54 EST
Wow.
Maybe we're experiencing some kind of DoS attack even if I think it's unlikely.

If this happens again it would be interesting whether the 100s of connections originate from a single IP or several.

I have no idea whether a max connections # > 1000 would be a good idea. Roger, maybe you can give us your opinion on this?
Comment 4 Denis Roy CLA 2014-02-05 13:39:52 EST
The connection count is already back up to 350 for port 1883 and it's climbing.  You can view connections using netstat -an | grep 1883
Comment 5 Benjamin Cabé CLA 2014-02-05 13:42:42 EST
FWIW we have already 222 ESTABLISHED connections to the MQTT broker just a few minutes after Denis did his magic.
So I guess the free eclipse.org broker is certainly very popular...
Comment 6 Denis Roy CLA 2014-02-05 13:46:22 EST
So do these "things" just connect, stay connected, and not transmit anything?  I don't see much bandwidth activity to/from your host.
Comment 7 Ian Skerrett CLA 2014-02-05 13:53:07 EST
(In reply to Benjamin CABE from comment #5)
> FWIW we have already 222 ESTABLISHED connections to the MQTT broker just a
> few minutes after Denis did his magic.
> So I guess the free eclipse.org broker is certainly very popular...

Any idea who or where the connections are coming from?
Comment 8 Roger Light CLA 2014-02-05 14:40:06 EST
1000 connections is unusual but not entirely unexpected. We are touting it as a sandbox server to try things out after all. I've had >4000 connections on test.mosquitto.org. In my experience they tend to stick around for a short while then disconnect.

You can see graphs of the stats here (I must rename it): https://xively.com/feeds/59871/

"active clients" is the plot of currently connected clients. You can get a view of the last 12 months with:

http://api.cosm.com/v2/feeds/59871/datastreams/1.png?b=true&g=true&w=360&h=200&duration=12month

From that you can see that 1000 connections or even close are rare.
Comment 9 Denis Roy CLA 2014-02-05 14:52:26 EST
Okay, I've increased max. connections to 10,000.  Enjoy
Comment 10 Roger Light CLA 2014-02-05 15:10:09 EST
I've been looking at the logs (/var/log/mosquitto/*) and in the last day, there have been connections from 2511 unique ip addresses using a client id of the form kong.protector/* and from 1077 unique ip addresses using a client id of the form location.konge/*

There is very little real data that looks like it comes from these clients but they do follow the bizarre practice of publishing to a keepalive topic that is implemented in the example android code at http://tokudu.com/post/50024574938/how-to-implement-push-notifications-for-android , so it looks like a phone app.