Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 302572 - Builds connecting to pserver at over 30 connections/second, causing pserver to limit rate.
Summary: Builds connecting to pserver at over 30 connections/second, causing pserver t...
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: CVS (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Eike Stepper CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-02-11 07:59 EST by Eike Stepper CLA
Modified: 2010-05-11 10:11 EDT (History)
6 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Eike Stepper CLA 2010-02-11 07:59:49 EST 
Together with Thomas I'm trying to make my new Buckminster build run in Hudson. But we discover frequent CVS problems:

     [java]       ERROR   Could not connect to :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling: Cannot connect to host: Connection refused

A full build log is permanently captured here:

     https://build.eclipse.org/hudson/view/Modeling/job/emf-cdo-integration/62/console

Is there anything we can do?
Comment 1 Eclipse Webmaster CLA 2010-02-11 09:33:00 EST 
You should be able to connect to pserver 'locally' on build ie: cvs -d /cvsroot/path/to/source .

I've taken a look, I don't see anything in the  system logs that would explain the error, and nobody else has reported anything, so I'm inclined to suspect somethings fishy in the build script.  

In looking at the build output is see this:

:pserver:anonymous@dev.eclipse.org:/cvsroot/modeling,org.eclipse.emf/org.eclipse.emf.net4j/doc/{0}

And I'm pretty sure the ',' should be a '/', or is that simply how the output is being formatted for the log?

-M.
Comment 2 Thomas Hallgren CLA 2010-02-11 09:40:10 EST 
(In reply to comment #1)
> You should be able to connect to pserver 'locally' on build ie: cvs -d
> /cvsroot/path/to/source .
> 
> I've taken a look, I don't see anything in the  system logs that would explain
> the error, and nobody else has reported anything, so I'm inclined to suspect
> somethings fishy in the build script.  
> 
The checkouts works intermittently. Sometimes everything is OK, then all of the sudden a bunch of "Connection refused". Spawn a new build and everything is OK again. No changes in the build script.

> In looking at the build output is see this:
> 
> :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling,org.eclipse.emf/org.eclipse.emf.net4j/doc/{0}
> 
> And I'm pretty sure the ',' should be a '/', or is that simply how the output
> is being formatted for the log?

That's Buckminsters notation of a repository with a module and it looks correct.
Comment 3 Denis Roy CLA 2010-02-11 09:56:27 EST 
> The checkouts works intermittently. Sometimes everything is OK, then all of the
> sudden a bunch of "Connection refused".

Here is what I'm seeing in my logs:
Feb 11 07:20:29 node2 xinetd[237976]: Deactivating service cvspserver due to excessive incoming connections.  Restarting in 10 seconds.

If I examine the pserver log prior to this, I see:

10/2/11@07:20:28: START: cvspserver pid=250753 from=172.25.25.34
10/2/11@07:20:28: EXIT: cvspserver status=0 pid=250752 duration=0(sec)
10/2/11@07:20:28: START: cvspserver pid=250756 from=172.25.25.34
10/2/11@07:20:28: START: cvspserver pid=250758 from=172.25.25.34
10/2/11@07:20:28: START: cvspserver pid=250760 from=172.25.25.34
10/2/11@07:20:28: EXIT: cvspserver status=0 pid=250753 duration=0(sec)
10/2/11@07:20:28: START: cvspserver pid=250762 from=172.25.25.34
10/2/11@07:20:28: EXIT: cvspserver status=0 pid=250756 duration=0(sec)
10/2/11@07:20:28: EXIT: cvspserver status=0 pid=250758 duration=0(sec)
10/2/11@07:20:28: START: cvspserver pid=250763 from=172.25.25.34
10/2/11@07:20:28: START: cvspserver pid=250765 from=172.25.25.34
10/2/11@07:20:28: EXIT: cvspserver status=0 pid=250760 duration=0(sec)
10/2/11@07:20:28: START: cvspserver pid=250767 from=172.25.25.34
10/2/11@07:20:28: EXIT: cvspserver status=0 pid=250762 duration=0(sec)
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250763 duration=1(sec)
10/2/11@07:20:29: START: cvspserver pid=250770 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250765 duration=1(sec)
10/2/11@07:20:29: START: cvspserver pid=250771 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250643 duration=5(sec)
10/2/11@07:20:29: START: cvspserver pid=250774 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250767 duration=1(sec)
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250774 duration=0(sec)
10/2/11@07:20:29: START: cvspserver pid=250776 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250776 duration=0(sec)
10/2/11@07:20:29: START: cvspserver pid=250778 from=172.25.25.34
10/2/11@07:20:29: START: cvspserver pid=250780 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250778 duration=0(sec)
10/2/11@07:20:29: START: cvspserver pid=250782 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250780 duration=0(sec)
10/2/11@07:20:29: START: cvspserver pid=250783 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250782 duration=0(sec)
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250783 duration=0(sec)
10/2/11@07:20:29: START: cvspserver pid=250786 from=172.25.25.34
10/2/11@07:20:29: START: cvspserver pid=250787 from=172.25.25.34
10/2/11@07:20:29: START: cvspserver pid=250789 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250786 duration=0(sec)
10/2/11@07:20:29: START: cvspserver pid=250791 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250787 duration=0(sec)
10/2/11@07:20:29: START: cvspserver pid=250794 from=172.25.25.34
10/2/11@07:20:29: EXIT: cvspserver status=0 pid=250789 duration=0(sec)
10/2/11@07:20:29: FAIL: cvspserver connections per second from=172.25.25.34

This failure is caused by exceeding 10 connections per *second*.  Creating and tearing down a connection causes a fair amount of strain on the server.

Another alternative is to use :local:/cvsroot/modeling instead, which is the local filesystem and does not require establishing a connection.  Your build may end up being faster.
Comment 4 Eike Stepper CLA 2010-02-11 11:16:48 EST 
Thanks for all the infos!

Could the too-many-connections problem be caused by Buckminster using multiple threads to materialize a workspace? Convenient in a local IDE, but maybe to heavy if forked by multiple server builds...
Comment 5 Thomas Hallgren CLA 2010-02-11 11:37:56 EST 
(In reply to comment #3)
> Another alternative is to use :local:/cvsroot/modeling instead, which is the
> local filesystem and does not require establishing a connection.  Your build
> may end up being faster.

What sounded like a great idea turned out to be impossible.

 bug 6892 "Support for local CVS repositories" is marked as a "Won't fix"

We will find other ways to limit the number of accesses.
Comment 6 Denis Roy CLA 2010-02-11 11:59:08 EST 
Is there no way of simply using the local filesystem? For instance, with command-line CVS, I can do this:

cvs -d /cvsroot/modeling co org.eclipse.....
Comment 7 Denis Roy CLA 2010-05-07 07:31:45 EDT 
Eike, is this still a problem?  If you're still hitting limits I can tweak them a bit.
Comment 8 Eike Stepper CLA 2010-05-07 09:12:03 EDT 
I didn't see them for a while. Closing as WORKSFORME for now and will reopen if it happens again. Thanks!
Comment 9 Eike Stepper CLA 2010-05-08 10:27:14 EDT 
Unfortunately I have to reopen this bug. Please see my latest build console:

https://build.eclipse.org/hudson/job/emf-cdo-integration/390/console
Comment 12 Denis Roy CLA 2010-05-09 14:31:28 EDT 
I looked at some logs, and found this:

May  9 14:08:33 node2 xinetd[123030]: Deactivating service cvspserver due to excessive incoming connections.  Restarting in 10 seconds.

Then I looked at the pserver log leading up to that time:


10/5/9@14:08:04: START: cvspserver pid=38718 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38720 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38721 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38724 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38726 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38728 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38730 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38732 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38733 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38736 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38738 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38739 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38742 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38743 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38745 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38747 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38750 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38752 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38754 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38755 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38758 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38759 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38762 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38764 from=172.25.25.34
10/5/9@14:08:04: START: cvspserver pid=38765 from=172.25.25.34
==second
10/5/9@14:08:05: START: cvspserver pid=38768 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38770 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38771 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38773 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38775 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38778 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38779 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38781 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38784 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38785 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38788 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38790 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38792 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38793 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38795 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38798 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38799 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38802 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38804 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38806 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38808 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38810 from=172.25.25.34
10/5/9@14:08:05: START: cvspserver pid=38812 from=172.25.25.34
==second
10/5/9@14:08:06: START: cvspserver pid=38814 from=172.25.25.34
10/5/9@14:08:06: START: cvspserver pid=38816 from=172.25.25.34
10/5/9@14:08:06: START: cvspserver pid=38818 from=172.25.25.34
10/5/9@14:08:06: START: cvspserver pid=38820 from=172.25.25.34
10/5/9@14:08:06: START: cvspserver pid=38822 from=172.25.25.34
10/5/9@14:08:06: START: cvspserver pid=38824 from=172.25.25.34
==second
10/5/9@14:08:07: START: cvspserver pid=38826 from=172.25.25.34
10/5/9@14:08:07: START: cvspserver pid=38828 from=172.25.25.34
10/5/9@14:08:07: START: cvspserver pid=38829 from=172.25.25.34
10/5/9@14:08:07: START: cvspserver pid=38831 from=172.25.25.34
10/5/9@14:08:07: START: cvspserver pid=38834 from=172.25.25.34
==second
10/5/9@14:08:08: START: cvspserver pid=38837 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38839 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38840 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38842 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38845 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38847 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38849 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38850 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38852 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38855 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38857 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38859 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38860 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38863 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38865 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38866 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38868 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38871 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38872 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38875 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38877 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38879 from=172.25.25.34
10/5/9@14:08:08: START: cvspserver pid=38880 from=172.25.25.34
==second
10/5/9@14:08:09: START: cvspserver pid=38883 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38885 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38886 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38888 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38890 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38893 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38894 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38897 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38898 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38901 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38902 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38905 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38907 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38909 from=172.25.25.34
10/5/9@14:08:09: START: cvspserver pid=38911 from=172.25.25.34
==second
10/5/9@14:08:11: START: cvspserver pid=38913 from=172.25.25.34
10/5/9@14:08:11: START: cvspserver pid=38915 from=172.25.25.34
10/5/9@14:08:11: START: cvspserver pid=38917 from=172.25.25.34
10/5/9@14:08:11: START: cvspserver pid=38919 from=172.25.25.34
10/5/9@14:08:11: START: cvspserver pid=38921 from=172.25.25.34
==second
10/5/9@14:08:14: START: cvspserver pid=38924 from=172.25.25.34
10/5/9@14:08:14: START: cvspserver pid=38928 from=172.25.25.34
==second
10/5/9@14:08:15: START: cvspserver pid=38930 from=172.25.25.34
10/5/9@14:08:15: START: cvspserver pid=38932 from=172.25.25.34
10/5/9@14:08:15: START: cvspserver pid=38934 from=172.25.25.34
10/5/9@14:08:15: START: cvspserver pid=38936 from=172.25.25.34
10/5/9@14:08:15: START: cvspserver pid=38937 from=172.25.25.34
==second
10/5/9@14:08:16: START: cvspserver pid=38940 from=172.25.25.34
10/5/9@14:08:16: START: cvspserver pid=38942 from=172.25.25.34
10/5/9@14:08:16: START: cvspserver pid=38944 from=172.25.25.34
10/5/9@14:08:16: START: cvspserver pid=38964 from=172.25.25.34
==second
10/5/9@14:08:17: START: cvspserver pid=38966 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38968 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38970 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38972 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38974 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38976 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38979 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38980 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38983 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38984 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38987 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38989 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38991 from=172.25.25.34
10/5/9@14:08:17: START: cvspserver pid=38992 from=172.25.25.34
==second
10/5/9@14:08:18: START: cvspserver pid=38997 from=172.25.25.34
10/5/9@14:08:18: START: cvspserver pid=38998 from=172.25.25.34
10/5/9@14:08:18: START: cvspserver pid=39000 from=172.25.25.34
10/5/9@14:08:18: START: cvspserver pid=39004 from=172.25.25.34
10/5/9@14:08:18: START: cvspserver pid=39007 from=172.25.25.34
10/5/9@14:08:18: START: cvspserver pid=39008 from=172.25.25.34
10/5/9@14:08:18: START: cvspserver pid=39011 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39013 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39014 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39018 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39020 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39023 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39024 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39027 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39029 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39031 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39032 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39035 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39037 from=172.25.25.34
10/5/9@14:08:19: START: cvspserver pid=39039 from=172.25.25.34
==second
10/5/9@14:08:20: START: cvspserver pid=39041 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39043 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39045 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39047 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39049 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39051 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39052 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39054 from=172.25.25.34
10/5/9@14:08:20: START: cvspserver pid=39057 from=172.25.25.34
==second
10/5/9@14:08:21: START: cvspserver pid=39059 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39061 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39063 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39065 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39067 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39069 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39071 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39073 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39075 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39077 from=172.25.25.34
10/5/9@14:08:21: START: cvspserver pid=39079 from=172.25.25.34
==second
10/5/9@14:08:22: START: cvspserver pid=39081 from=172.25.25.34
10/5/9@14:08:22: START: cvspserver pid=39083 from=172.25.25.34
10/5/9@14:08:22: START: cvspserver pid=39085 from=172.25.25.34
10/5/9@14:08:22: START: cvspserver pid=39087 from=172.25.25.34
10/5/9@14:08:22: START: cvspserver pid=39089 from=172.25.25.34
10/5/9@14:08:22: START: cvspserver pid=39091 from=172.25.25.34
10/5/9@14:08:22: START: cvspserver pid=39093 from=172.25.25.34
10/5/9@14:08:22: START: cvspserver pid=39095 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39098 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39099 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39101 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39103 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39106 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39108 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39110 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39111 from=172.25.25.34
10/5/9@14:08:23: START: cvspserver pid=39113 from=172.25.25.34
==second
10/5/9@14:08:24: START: cvspserver pid=39116 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39118 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39119 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39122 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39124 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39125 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39128 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39129 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39132 from=172.25.25.34
10/5/9@14:08:24: START: cvspserver pid=39134 from=172.25.25.34
==second
10/5/9@14:08:25: START: cvspserver pid=39139 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39141 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39143 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39145 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39146 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39149 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39151 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39153 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39154 from=172.25.25.34
10/5/9@14:08:25: START: cvspserver pid=39157 from=172.25.25.34
==second
10/5/9@14:08:26: START: cvspserver pid=39158 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39161 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39163 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39165 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39167 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39169 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39171 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39173 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39175 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39177 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39179 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39181 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39183 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39185 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39187 from=172.25.25.34
10/5/9@14:08:26: START: cvspserver pid=39189 from=172.25.25.34
==second
10/5/9@14:08:27: START: cvspserver pid=39191 from=172.25.25.34
10/5/9@14:08:27: START: cvspserver pid=39193 from=172.25.25.34
10/5/9@14:08:27: START: cvspserver pid=39195 from=172.25.25.34
10/5/9@14:08:27: START: cvspserver pid=39197 from=172.25.25.34
10/5/9@14:08:27: START: cvspserver pid=39199 from=172.25.25.34
==second
10/5/9@14:08:28: START: cvspserver pid=39202 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39203 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39206 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39208 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39210 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39211 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39214 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39215 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39217 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39219 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39222 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39223 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39226 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39228 from=172.25.25.34
10/5/9@14:08:28: START: cvspserver pid=39230 from=172.25.25.34
==second
10/5/9@14:08:29: START: cvspserver pid=39232 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39234 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39236 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39237 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39239 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39241 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39243 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39246 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39247 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39249 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39252 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39254 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39255 from=172.25.25.34
10/5/9@14:08:29: START: cvspserver pid=39258 from=172.25.25.34
==second
10/5/9@14:08:30: START: cvspserver pid=39260 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39261 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39263 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39265 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39268 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39269 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39272 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39274 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39276 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39278 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39279 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39281 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39284 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39286 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39288 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39290 from=172.25.25.34
10/5/9@14:08:30: START: cvspserver pid=39292 from=172.25.25.34
==second
10/5/9@14:08:31: START: cvspserver pid=39294 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39296 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39298 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39299 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39302 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39303 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39305 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39307 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39309 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39312 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39314 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39316 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39318 from=172.25.25.34
10/5/9@14:08:31: START: cvspserver pid=39319 from=172.25.25.34
==second
10/5/9@14:08:32: START: cvspserver pid=39321 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39324 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39325 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39328 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39330 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39332 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39333 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39336 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39338 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39339 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39342 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39344 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39346 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39348 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39350 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39351 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39354 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39356 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39358 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39360 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39362 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39364 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39366 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39368 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39370 from=172.25.25.34
10/5/9@14:08:32: START: cvspserver pid=39371 from=172.25.25.34
==second
10/5/9@14:08:33: START: cvspserver pid=39375 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39376 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39379 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39381 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39382 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39385 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39386 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39389 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39391 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39393 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39394 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39396 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39399 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39401 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39403 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39405 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39407 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39408 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39411 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39412 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39414 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39417 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39419 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39421 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39422 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39425 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39427 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39429 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39431 from=172.25.25.34
10/5/9@14:08:33: START: cvspserver pid=39432 from=172.25.25.34

^^ 30 connections per second.


Your build is connecting to pserver at over 30 connections per second, which is the service limit.  Of course, I could increase the limit, but a) I can't help but think this is a bit excessive and b) it will slow pserver down because of the heavy CPU usage, at which point you'll be opening a bug saying pserver is slow.

So my question to you is: why is your build establishing such a rapid rate of connections?
Comment 13 Denis Roy CLA 2010-05-09 14:35:01 EDT 
Another alternative to users of build.eclipse.org is to use local: instead of pserver:
Comment 14 Thomas Hallgren CLA 2010-05-09 14:43:06 EDT 
(In reply to comment #13)
> Another alternative to users of build.eclipse.org is to use local: instead of
> pserver:

That's not an option I'm afraid, since local: isn't supported by the Eclipse CVS team support.
Comment 15 Thomas Hallgren CLA 2010-05-09 14:44:49 EDT 
(In reply to comment #12)
> Your build is connecting to pserver at over 30 connections per second, which is
> the service limit.

Eike, we had this problem before. The cure was to improve the RMAP to limit the number of false positives. Did you revert that so that you now have search paths with a large number of providers again?
Comment 16 Denis Roy CLA 2010-05-09 14:46:59 EDT 
Allright, I increased it to 200 connections per second.  Beat it up good, guys!
Comment 17 James Blackburn CLA 2010-05-09 15:05:24 EDT 
(In reply to comment #14)
> That's not an option I'm afraid, since local: isn't supported by the Eclipse
> CVS team support.

The other thing you can do is:
:extssh:build_user@localhost:/path/to/cvs_rep

The catch is that it requires your build_user to be able to ssh locally, but if you're willing to fake up an account with read access to the shard CVS mounts, this could do the trick for you.
Comment 18 Ed Willink CLA 2010-05-09 17:13:26 EDT 
It is not clear to me where the above comments apply.

Should MAP files be changed to not use pserver?
Comment 19 Denis Roy CLA 2010-05-09 19:38:28 EDT 
(In reply to comment #18)
> Should MAP files be changed to not use pserver?


Only if you're establishing more than 30 connections per second to dev.eclipse.org.
Comment 20 Eike Stepper CLA 2010-05-10 03:54:07 EDT 
My Buckminster RMAP has not changed, at least not the CVS search paths: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.cdo/releng/org.eclipse.emf.cdo.releng/build.rmap?root=Modeling_Project&view=markup

Correct me if I'm wrong, but that doesn't seem to be the problem. Maybe it's the increasing number of Bucky-based Hudson jobs, each using 4 concurrent resolver threads?

:local: transport would be ideal, but, as Thomas pointed out twice in comment #5 and comment #14, this is not an option.

Even more ideal would be to just reuse the checkouts that Hudson has already provided in the Hudson workspace. Everything I need is already there. Unfortunately our CVS module layout is not very suitable for this because we didn't have this issue in mind when we designed this layout 6 years back ;-(

Thomas, do you think it would be worth it nevertheless?
Comment 21 Eike Stepper CLA 2010-05-11 02:53:52 EDT 
Now I get a different error:

    Connection to dev.eclipse.org timed out while waiting for another thread 
    to make a connection to the same host

And the provisioning phase of the build is still failing (randomly):

    https://build.eclipse.org/hudson/job/emf-cdo-integration/401/console
Comment 22 Denis Roy CLA 2010-05-11 09:39:22 EDT 
I've seen nothing in my logs that would indicate a service interruption for pserver coming from the build server for May 10.  I think your build is haunted.
Comment 23 Eike Stepper CLA 2010-05-11 10:11:23 EDT 
Yes ;-(

And I even can't check the other logs because of this JellyTagException.