Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 328334 - Hudson slave 1 SSH failures
Summary: Hudson slave 1 SSH failures
Status: CLOSED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: CI-Jenkins (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Eclipse Webmaster CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-10-21 06:27 EDT by Glyn Normington CLA
Modified: 2010-10-21 12:25 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Glyn Normington CLA 2010-10-21 06:27:24 EDT
Various Virgo jobs are failing very early on slave 1.

For example, see https://hudson.eclipse.org/hudson/view/Virgo/job/virgo.util.snapshot/. Job 132 failed on slave 1, whereas job 133 passed on slave 2.

The failure shows the following console log:

Started by an SCM change
Building remotely on hudson-slave1
hudson.util.IOException2: remote file operation failed: /opt/users/hudsonbuild/workspace/virgo.util.snapshot at hudson.remoting.Channel@782931c7:hudson-slave1
	at hudson.FilePath.act(FilePath.java:749)
	at hudson.FilePath.act(FilePath.java:735)
	at hudson.FilePath.mkdirs(FilePath.java:801)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1059)
	at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479)
	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411)
	at hudson.model.Run.run(Run.java:1273)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:129)
Caused by: java.io.IOException: SSH channel is closed. (Close requested by remote)
	at com.trilead.ssh2.channel.ChannelManager.sendData(ChannelManager.java:383)
	at com.trilead.ssh2.channel.ChannelOutputStream.write(ChannelOutputStream.java:63)
	at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1838)
	at java.io.ObjectOutputStream$BlockDataOutputStream.writeByte(ObjectOutputStream.java:1876)
	at java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1537)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:329)
	at hudson.remoting.Channel.send(Channel.java:419)
	at hudson.remoting.Request.call(Request.java:105)
	at hudson.remoting.Channel.call(Channel.java:557)
	at hudson.FilePath.act(FilePath.java:742)
	... 9 more
Archiving artifacts
ERROR: Failed to archive artifacts: build-util/target/
java.io.IOException: SSH channel is closed. (Close requested by remote)
	at com.trilead.ssh2.channel.ChannelManager.sendData(ChannelManager.java:383)
	at com.trilead.ssh2.channel.ChannelOutputStream.write(ChannelOutputStream.java:63)
	at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1838)
	at java.io.ObjectOutputStream$BlockDataOutputStream.writeByte(ObjectOutputStream.java:1876)
	at java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1537)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:329)
	at hudson.remoting.Channel.send(Channel.java:419)
	at hudson.remoting.Request.call(Request.java:105)
	at hudson.remoting.Channel.call(Channel.java:557)
	at hudson.EnvVars.getRemote(EnvVars.java:202)
	at hudson.model.Computer.getEnvironment(Computer.java:745)
	at hudson.model.Run.getEnvironment(Run.java:1673)
	at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:682)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:116)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
	at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:580)
	at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:558)
	at hudson.model.Build$RunnerImpl.post2(Build.java:158)
	at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:528)
	at hudson.model.Run.run(Run.java:1296)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:129)
Recording test results
ERROR: Publisher hudson.tasks.junit.JUnitResultArchiver aborted due to exception
java.io.IOException: SSH channel is closed. (Close requested by remote)
	at com.trilead.ssh2.channel.ChannelManager.sendData(ChannelManager.java:383)
	at com.trilead.ssh2.channel.ChannelOutputStream.write(ChannelOutputStream.java:63)
	at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1838)
	at java.io.ObjectOutputStream$BlockDataOutputStream.writeByte(ObjectOutputStream.java:1876)
	at java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1537)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:329)
	at hudson.remoting.Channel.send(Channel.java:419)
	at hudson.remoting.Request.call(Request.java:105)
	at hudson.remoting.Channel.call(Channel.java:557)
	at hudson.EnvVars.getRemote(EnvVars.java:202)
	at hudson.model.Computer.getEnvironment(Computer.java:745)
	at hudson.model.Run.getEnvironment(Run.java:1673)
	at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:682)
	at hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:130)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
	at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:580)
	at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:558)
	at hudson.model.Build$RunnerImpl.post2(Build.java:158)
	at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:528)
	at hudson.model.Run.run(Run.java:1296)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:129)
ERROR: Publisher hudson.tasks.Mailer aborted due to exception
java.io.IOException: SSH channel is closed. (Close requested by remote)
	at com.trilead.ssh2.channel.ChannelManager.sendData(ChannelManager.java:383)
	at com.trilead.ssh2.channel.ChannelOutputStream.write(ChannelOutputStream.java:63)
	at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1838)
	at java.io.ObjectOutputStream$BlockDataOutputStream.writeByte(ObjectOutputStream.java:1876)
	at java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1537)
	at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:329)
	at hudson.remoting.Channel.send(Channel.java:419)
	at hudson.remoting.Request.call(Request.java:105)
	at hudson.remoting.Channel.call(Channel.java:557)
	at hudson.EnvVars.getRemote(EnvVars.java:202)
	at hudson.model.Computer.getEnvironment(Computer.java:745)
	at hudson.model.Run.getEnvironment(Run.java:1673)
	at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:682)
	at hudson.tasks.Mailer.perform(Mailer.java:108)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:601)
	at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:580)
	at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:558)
	at hudson.model.Build$RunnerImpl.post2(Build.java:158)
	at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:528)
	at hudson.model.Run.run(Run.java:1296)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:129)
Finished: FAILURE
Comment 1 Eclipse Webmaster CLA 2010-10-21 09:44:45 EDT
I've restarted the slave.

-M.
Comment 2 Glyn Normington CLA 2010-10-21 10:01:57 EDT
Thanks Matt.

For future reference, what led you to believe that a restart would help? And can I restart a slave if this recurs or will I have to bother you guys?
Comment 3 Eclipse Webmaster CLA 2010-10-21 10:47:18 EDT
A similar error was reported in another bug(regarding the ssh channel) and our digging indicated the the solution was to disconnect and reconnect the slave.

It's something you'll either need to file a bug or send an email for.

-M.
Comment 4 Glyn Normington CLA 2010-10-21 10:53:02 EDT
Ok, thanks.
Comment 5 Steve Powell CLA 2010-10-21 12:25:19 EDT
This time...