Community
Participate
Working Groups
If one connects to an SSH sever which has MaxSession set to less than 4 than opening the connection fails with "Opening SFTP connection failed. Does the remote machine support SFTP?". The reason is that org.eclipse.ptp.remotetools.internal.ssh.Connection.connect(IAuthInfo, String, int, String, int, IProgressMonitor) opens 3 SFTP channels (after opening 1 control channel). We could reduce the pool size but I think it is very unlikely to encounter servers with anything between 1 and 10 (default). Either the admin doesn't want additional sessions (than he would set it to 1) or he doesn't have any good reason to change the default. While it would be very nice to support MaxSession=1 (NCCS recently changed their SSH server to that), I think it would be extremely hard to support. This would require to always close sessions as soon as the are done and waiting with any operation until the one session slot is free. We probably should fix the error message. Do we require to test what the reason is? Or do we just want to make the error message: "Opening SFTP connection failed. Does the remote machine support SFTP and multiple sessions? MaxSessions has to be at least 4."
Why are there 3 SFTP channels? Could this be reduced to 1? The new error message seems fine.
We have more than one SFTP connection for higher performance. This allows two threads to transfer files in parallel. We had seen this is used for 2 threads. I'm not sure how often it is used for 3. So we probably could reduce it to 2. But I really doubt that any server has anything else than 1 or 10 so it shouldn't matter whether we have 1, 2, or 3 in the pool. Committed the error message. If you agree that we cannot make it work for MaxSessions=1 or 2 (even with pool=1 we need 3 channels (execution, control, sftp) than I suppose we should close it as WONTFIX
NCCS will not allow any workaround for the multiplexing. Thus we have to try to make it work without multiplexing. Some ideas what to do if multiplexing is not available - Make the SFTP pool detect the error and use pool-size 1 - If multiplexing is not available open a new connection for each channel (prompting for a SecurID for a connection requiring it) - Create the sftp connection on demand so that the user isn't prompted for a SecureID when no sftp connection is required - Ideally use something similar to GNU screen to run the control and execution commands over the same connection Questions: - Is something similar to GNU screen available which we could use to run several commands over one SSH channel - Does JSch work correctly with MaxSession=1 as long as only one channel is opened? Does it work to open a new ExecChannel after the previous command finished?
I don't understand the comment for org.eclipse.ptp.remotetools.internal.ssh.Connection.createExecChannel. It says * Channels without PTY may be managed, by * it is recommended no to be. This will allocate them into the default * session, since an unlimited number of not PTY channels can be allocated * into the default session. Why would the number of channels with PTY be limtied but the number of channels without PTY be unlimited? Is there some SSH server with this odd configuration?
(In reply to comment #3) > NCCS will not allow any workaround for the multiplexing. Thus we have to try to > make it work without multiplexing. > > Some ideas what to do if multiplexing is not available > - Make the SFTP pool detect the error and use pool-size 1 > - If multiplexing is not available open a new connection for each channel > (prompting for a SecurID for a connection requiring it) > - Create the sftp connection on demand so that the user isn't prompted for a > SecureID when no sftp connection is required > - Ideally use something similar to GNU screen to run the control and execution > commands over the same connection > > Questions: > - Is something similar to GNU screen available which we could use to run > several commands over one SSH channel > - Does JSch work correctly with MaxSession=1 as long as only one channel is > opened? Does it work to open a new ExecChannel after the previous command > finished? I could imaging reducing the number of channels required to 2, one for control/exec, and one for sftp. The sftp channel could be created on a separate session on demand as you suggest, possibly requiring a second authentication. There is nothing requiring only a single command per exec channel. Once a channel is open, you can send as many commands as you like. I think it's only done this way for simplicity, such as knowing when one command finishes so another can be started, etc. I'm not sure you'd want to use the screen protocol as it provides a whole lot of stuff, like the ability to disconnect/reconnect, that is not required. This would all need to be implemented as a special case when MaxSession is 1. It seems like a large piece of work to me.
(In reply to comment #4) > I don't understand the comment for > org.eclipse.ptp.remotetools.internal.ssh.Connection.createExecChannel. > > It says > * Channels without PTY may be managed, by > * it is recommended no to be. This will allocate them into the default > * session, since an unlimited number of not PTY channels can be allocated > * into the default session. > > Why would the number of channels with PTY be limtied but the number of channels > without PTY be unlimited? Is there some SSH server with this odd configuration? I presume this is because the total number of PTYs on the system may be limited. It used to be 1024 on Linux systems, but I'm not sure what it is these days.
(In reply to comment #5) > I could imaging reducing the number of channels required to 2, one for > control/exec, and one for sftp. How would you combine control and exec? We have been thinking about removing the need of the control channel. As far as we understand the only function of the control channel is to be able to kill a command. Thus if we make the shell, which is runnig the command, react to either closing the ssh connection or sending some command sequence, to kill the command, than we wouldn't need the extra control channel. It is possible to use only one channel by creating the sftp channel just for one sftp command and closing it afterwards. That way the stftp channel and exec channel could be interleaved over the same session. But that probably hurts performance to much for being a useful solution. > There is nothing requiring only a single command per exec channel. Once a > channel is open, you can send as many commands as you like. But you can only send the command sequentially. If you want to send commands in parallel you do need a 2nd channel, right? > I'm not sure you'd want to use the screen protocol > as it provides a whole lot of stuff, like the ability to disconnect/reconnect, > that is not required. No I wouldn't want to use the screen protocol. I was thinking something similar which offers the option to run more than one command over one ssh channel in parallel. It could be as simple as a script running commands in parallel, and prints the stdout, stderr, and exit value prefixed with an ID for the command. This way the commands can run in parallel and we still get the output and exit value of each of the commands. > This would all need to be implemented as a special case when MaxSession is 1. > It seems like a large piece of work to me. Some of the changes (like removing the control channel if not needed anyhow) might also be beneficial in other cases. But I agree to make this work nicely it requires quite a bit of extra work. I have a prototype working which works (creating sync project, syncing, building running) on 3 ssh session (requiring 3 tokens). It still sometimes has deadlocks but those seem to be not specific to MaxSession=1 and should be fixed anyhow. The only real thing I changed is: - SFTP_POOLSIZE = 1 - maxChannelsPerConnection = 1 - initialDefaultSessionLoad = 1 - always use the connection pool (including for the control channel just not for sftp) So it is somewhat working. But obviously entering 3 tokens is very annyoing. Also ideally we would autodetect the value of MaxSession so that we can set these settings automatically correct. If not possible we need to make that a user configuration for the connection.
(In reply to comment #6) > (In reply to comment #4) > > Why would the number of channels with PTY be limtied but the number of channels > > without PTY be unlimited? Is there some SSH server with this odd configuration? > > I presume this is because the total number of PTYs on the system may be > limited. It used to be 1024 on Linux systems, but I'm not sure what it is these > days. That would make sense. But RemoteTools doesn't limit the number of total PTYs per remote host but instead the number of PTYs per SSH session/connection. If more than 6 commands, requiring a PTY, are executed in parallel than the code currently opens a new connection and executes further commands on this 2nd connection. Thus it never allocates more than 6 PTYs per connection but the total number is not limited. I don't see how this would make any sense.
Created attachment 206370 [details] Makes PTP work with MaxSession=1 - still requires 3 passwords
I just tried MaxSessions=1 in Luna and it seemed to work without requiring any additional passwords (I have ssh key set up).
If more commands are executed in parallel than MaxSessions than some of the commands fail. This is because only the sftp channel gets its own session in that case. All exec channels try to use the same channel. I think a solution could: We add a usage counter per session. When getExecChannel is called it is incremented. If it is equal to MaxSessions than we open a new session and use it to create the exec channel. When the command was executed the usage counter is decremented again. There doesn't seem to be a way to query MaxSessions. It probably would work to assume it to be 10 unless we notice during open that it is 1 (using the current approach).
Remote tools is no longer supported. A new bug should be opened against the Remote component if this is still a problem.
