Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 365551

Summary:	windows build slave "windows7tests" has very frequent network timeouts
Product:	Community	Reporter:	Jan Sievers <jan.sievers>
Component:	CI-Jenkins	Assignee:	Eclipse Webmaster <webmaster>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	P3	CC:	igor, t-oberlies
Version:	unspecified
Target Milestone:	---
Hardware:	PC
OS:	Windows 7
Whiteboard:

Description Jan Sievers

2011-12-05 03:28:03 EST

we are using the windows test slave to execute the tycho integration tests.

Throughout the last 2 months we found that we get a very high number of false positive test failures because of intermittent network failures.

See the build list
https://hudson.eclipse.org/hudson/job/tycho-its-win-nightly/

almost every test failure is caused by network connection timeout, typical error messge:

[INFO] Resolving target platform for project MavenProject: org.eclipse.tycho:org.eclipse.tycho.p2.resolver.facade:0.14.0-SNAPSHOT @ C:\hb\workspace\tycho-its-win-nightly\tycho-bundles\org.eclipse.tycho.p2.resolver.facade\pom.xml
Nov 15, 2011 11:19:00 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:19:00 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:19:21 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:19:21 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:19:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:19:42 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:20:25 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:20:25 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:20:46 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:20:46 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:21:07 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:21:07 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:21:49 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:21:49 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:22:10 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:22:10 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:22:31 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:22:31 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:23:13 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:23:13 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:23:34 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:23:34 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
Nov 15, 2011 11:23:55 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
Nov 15, 2011 11:23:55 PM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
INFO: Retrying request
[ERROR] Internal error: java.lang.RuntimeException: Failed to resolve target definition C:\hb\workspace\tycho-its-win-nightly\tycho-bundles\tycho-bundles-target\build.target: Failed to load metadata repository from location http://download.eclipse.org/eclipse/updates/3.8milestones/S-3.8M1-201108031800/: Unable to connect to repository http://download.eclipse.org/eclipse/updates/3.8milestones/S-3.8M1-201108031800/content.xml: Connection timed out: connect -> [Help 1]



For comparison, the same tests are triggered at the same time on linux and are mostly fine:

https://hudson.eclipse.org/hudson/job/tycho-its-linux-nightly/

Any hints what's going on here?

Thanks
Jan

Comment 1 Denis Roy

2011-12-12 14:31:45 EST

Are you still seeing this error?

Comment 2 Jan Sievers

2011-12-13 02:28:20 EST

(In reply to comment #1)
> Are you still seeing this error?

yes. Last failed build is from yesterday
https://hudson.eclipse.org/hudson/job/tycho-its-win-nightly/91/console

As I said the linux machine executing the same tests is fine
https://hudson.eclipse.org/hudson/job/tycho-its-linux-nightly/

Comment 3 Eclipse Webmaster

2011-12-14 14:11:17 EST

Interesting, it appears the Windows slave was getting DNS results in a different order then the other Slaves.  I've tweaked the DNS server to order the results correctly and Windows seems happy now(I also added the correct IP to the hosts file just to be on the safe side).

-M.

Comment 4 Jan Sievers

2011-12-15 03:30:13 EST

(In reply to comment #3)
> Interesting, it appears the Windows slave was getting DNS results in a
> different order then the other Slaves.  I've tweaked the DNS server to order
> the results correctly and Windows seems happy now(I also added the correct IP
> to the hosts file just to be on the safe side).
> 
> -M.

thanks!

Things look better now on windows7tests [1]. I will wait for the next couple of builds and then resolve this bug if everything stays fine.

[1] https://hudson.eclipse.org/hudson/job/tycho-its-win-nightly/

Comment 5 Jan Sievers

2011-12-16 08:43:42 EST

4 succeeded builds in a row