1 <html> 2 <title>Ganymed SSH-2 for Java FAQ</title> 3 <body> 4 5 <a name="oben"></a> 6 <h1>Ganymed SSH-2 for Java FAQ</h1> 7 8 <p> 9 This FAQ includes information regarding topics that were discussed in e-mails between developers and users 10 of the Ganymed SSH-2 for Java library. 11 </p> 12 <p> 13 Ganymed SSH-2 for Java homepage: <a href="http://www.cleondris.ch/opensource/ssh2/">http://www.cleondris.ch/opensource/ssh2/</a><br> 14 Last update of FAQ: apr-16-2010. 15 </p> 16 <p> 17 Please report bugs, typos and any kind of suggestions to Christian Plattner (christian.plattner at cleondris.ch). 18 </p> 19 20 <hr> 21 22 <h2>Sections:</h2> 23 24 <p> 25 <ul> 26 <li><a href="#env">When I start program XYZ with putty (or openssh, ..., whatever) then everything works. 27 However, if I use "Session.execCommand", then XYZ behaves differently or does not work at all!</a></li> 28 29 <li><a href="#blocking">My program sometimes hangs when I only read output from stdout! 30 Or: can you explain me the story about the shared stdout/stderr window in the SSH-2 protocol? 31 Or: what is this "StreamGobbler" thing all about?</a></li> 32 33 <li><a href="#buffered">Why are the session's Input- and OutputStreams not buffered?</a></li> 34 35 <li><a href="#sessioncommands">Why can't I execute several commands in one single session?</a></li> 36 37 <li><a href="#sessionlimit">I cannot open more than 10 concurrent sessions (or SCP clients).</a></li> 38 39 <li><a href="#passwordauth">Password authentication fails, I get "Authentication method password not 40 supported by the server at this stage".</a></li> 41 42 <li><a href="#puttygen">Why does public key authentication fail with my putty key?</a></li> 43 44 <li><a href="#catmethod">I am sending data to a remote file using the "cat" method, but not all data is being written.</a></li> 45 46 <li><a href="#pumptoremote">I want to pump data into a remote file, but the amount of data to be sent 47 is not known at the time the transfer starts.</a></li> 48 49 <li><a href="#swingshell">Do you have an example for the usage of feature XYZ?</a></li> 50 51 <li><a href="#maven">Where is the official Maven repository?</a></li> 52 53 </ul> 54 </p> 55 56 <hr><a name="env"></a><h2>When I start program XYZ with putty (or openssh, ..., whatever) then everything 57 works. However, if I use "Session.execCommand", then XYZ behaves differently or does not work at all!</h2> 58 59 <h3>Short answer:</h3> 60 61 <p> 62 The most often source of problems when executing a command with <tt>Session.execCommand()</tt> 63 are missing/wrong set environment variables on the remote machine. Make sure that the minimum needed 64 environment for XYZ is the same, independentely on how the shell is being invoked. 65 </p> 66 67 <p> 68 Example quickfix for bash users: 69 </p> 70 71 <p> 72 <ol> 73 <li>Define all your settings in the file <tt><b>~/.bashrc</b></tt></li> 74 <li>Make sure that the file <tt><b>~/.bash_profile</b></tt> only contains the line <tt><b>source 75 ~/.bashrc</b></tt>.</li> 76 <li>Before executing <tt>Session.execCommand()</tt>, do NOT aquire any type of pseudo terminal in the 77 session. Be prepared to consume stdout and stderr data.</li> 78 </ol> 79 </p> 80 81 <p> 82 <b>Note:</b> If you really want to mimic the behavior of putty, then don't use Session.execCommand(), 83 instead aquire a pty (pseudo terminal) and then start a shell (use <tt>Session.requestPTY()</tt> and 84 <tt>Session.startShell()</tt>). You then have to communicate with the shell process at the other end 85 through stdin and stdout. However, you also have to implement terminal logic (e.g., escape sequence 86 handling (unless you use a "dumb" pty), "expect-send" logic (output parsing, shell prompt detection), etc.). 87 </p> 88 89 <h3>Long answer:</h3> 90 91 <p> 92 If you login by using putty, then putty will normally request a "xterm" pty and your assigned shell 93 (e.g., bash) will be started (a so called "interactive login shell"). In contrast, if you use 94 <tt>Session.execCommand()</tt> to start a command then (unless you ask for it) no pty will be aquired 95 and the command will be given to the shell as an argument (with the shell's "-c" option). 96 </p> 97 98 <p> 99 The way a shell is being invoked has an effect on the set of initialization files which will be read be the shell. 100 </p> 101 102 <p> 103 To demonstrate the difference, try the following (from the command line, e.g., with an OpenSSH client): 104 </p> 105 106 <p> 107 <ol> 108 <li>Login interactively and print the environment with the "env" command:<br> <br> 109 <tt><b>[user@host ~] ssh 127.0.0.1<br> 110 [user@host ~] env</b></tt><br> <br> 111 </li> 112 <li>Let the ssh server execute the "env" command (equivalent to using <tt>Session.executeCommand()</tt>):<br> <br> 113 <tt><b>[user@host ~] ssh 127.0.0.1 "env"</b></tt> 114 </li> 115 </ol> 116 </p> 117 118 <p> 119 If you compare the two outputs, then you will (unless you have adjusted your shell's settings) 120 observe different environments. 121 </p> 122 123 <p> 124 <b>If you are interested in the details, then please read the <tt>INVOCATION</tt> section in man page 125 for the bash shell. You may notice that the definitions of "interactive" and "non-interactive" 126 (and combinations with "login") are little bit tricky.</b> 127 </p> 128 129 [<a href="#oben">TOP</a>] 130 131 <hr><a name="blocking"></a><h2>My program sometimes hangs when I only read output from stdout! 132 Or: can you explain me the story about the shared stdout/stderr window in the SSH-2 protocol? 133 Or: what is this "StreamGobbler" thing all about?</h2> 134 135 <p> 136 In the SSH-2 low level protocol, each channel (e.g., session) has a receive window. When the remote 137 SSH daemon has filled up our receive window, it must wait until we have consumed the input and are ready to accept new data. 138 </p> 139 140 <p> 141 Unfortunately, the SSH-2 protocol defines a shared window for stderr and stdout. As a consequence, 142 if, for example, the remote process produces a lot of stderr data and you never consume it, then after 143 some time the local receive window will be full and the sender is blocked. If you then try to read() 144 from stdout, your call will be blocked: there is no stdout data (locally) available and the SSH daemon 145 cannot send you any, since the receive window is full (you would have to read some stderr data first 146 to "free" up space in the receive window). 147 </p> 148 149 <p> 150 Fortunately, Ganymed SSH-2 uses a 30KB window - the above described scenario should be very rare. 151 </p> 152 153 <p> 154 Many other SSH-2 client implementations just blindly consume any remotely produced data into a buffer 155 which gets automatically extended - however, this can lead to another problem: in the extreme case 156 the remote side can overflow you with data (e.g., leading to out of memory errors). 157 </p> 158 159 <p> 160 What can you do about this? 161 </p> 162 163 <p> 164 <ol> 165 <li><b>Bad: Do nothing</b> - just work with stderr and stdout Inputstreams and hope that the 30KB 166 window is enough for your application.</li> 167 168 <li><b>Better, recommended for most users:</b> use two worker threads that consume remote stdout 169 and stderr in parallel. Since you probably are not in the mood to program such a thing, you can use 170 the StreamGobbler class supplied with Ganymed SSH-2. The Streamgobbler is a special InputStream that 171 uses an internal worker thread to read and buffer internally all data produced by another InputStream. 172 It is very simple to use:<br> <tt><b><pre>InputStream stdout = new StreamGobbler(mysession.getStdout()); 173 174 InputStream stderr = new StreamGobbler(mysession.getStderr());</pre></b></tt> 175 You then can access stdout and stderr in any order, in the background the StreamGobblers will 176 automatically consume all data from the remote side and store in an internal buffer.</li> 177 178 <li><b>Advanced:</b> you are paranoid and don't like programs that automatically extend buffers 179 without asking you. You then have to implement a state machine. The condition wait facility offered by 180 <tt>Session.waitForCondition()</tt> is exactly what you need: you can use it to wait until either stdout 181 or stderr data has arrived and can be consumed with the two InputStreams. You can either use the return value 182 of <tt>Session.waitForCondition()</tt> or check with <tt>InputStream.available()</tt> 183 (for stdout and stderr) which InputStream has data available (i.e., a <tt>read()</tt> call will not block). 184 Be careful when wrapping the InputStreams, also do not concurrently call read() on the InputStreams while calling 185 <tt>Session.waitForCondition()</tt> (unless you know what you are doing).<br>Please have a look a the 186 <tt>SingleThreadStdoutStderr.java</tt> example.</li> 187 188 <li><b>The lazy way:</b> you don't mind if stdout and stderr data is being mixed into the same 189 stream. Just allocate a "dumb" pty and the server will hopefully not send you any data on the stderr 190 stream anymore. <b>Note:</b> by allocating a pty, the shell used to execute the command will probably 191 behave differently in terms of initialization (see also <a href="#env">this question</a>).</li> 192 </ol> 193 </p> 194 195 196 [<a href="#oben">TOP</a>] 197 198 <hr><a name="buffered"></a><h2>Why are the session's Input- and OutputStreams not buffered?</h2> 199 200 <p> 201 If you need it, then this library offers quite a raw type of access to the SSH-2 protocol stack. 202 Of course, many people don't need that kind of low level access. If you need buffered streams, 203 then you should the do the same thing as you would probably do with the streams of a TCP socket: 204 wrap them with instances of BufferedInputStream and BufferedOutputStream. In case you use 205 StreamGobblers for the InputStreams, then you don't need any additional wrappers, since the 206 StreamGobblers implement buffering already. 207 </p> 208 <p> 209 This code snippet will probably work well for most people: 210 </p> 211 <p> 212 <tt> 213 <pre> 214 InputStream stdout = new StreamGobbler(mysession.getStdout()); 215 InputStream stderr = new StreamGobbler(mysession.getStderr()); 216 OutputStream stdin = new BufferedOutputStream(mysession.getStdin(), 8192); 217 </pre> 218 </tt> 219 </p> 220 221 [<a href="#oben">TOP</a>] 222 223 <hr><a name="sessioncommands"></a><h2>Why can't I execute several commands in one single session?</h2> 224 <p> 225 If you use <tt>Session.execCommand()</tt>, then you indeed can only execute only one command per session. 226 This is not a restriction of the library, but rather an enforcement by the underlying SSH-2 protocol 227 (a <tt>Session</tt> object models the underlying SSH-2 session). 228 </p> 229 <p> 230 There are several solutions: 231 </p> 232 <p> 233 <ul> 234 <li><b>Simple: Execute several commands in one batch</b>, e.g., something like <tt>Session.execCommand("echo 235 Hello && echo again")</tt>.</li> 236 <li><b>Simple: The intended way: simply open a new session for each command</b> - once you have opened a 237 connection, you can ask for as many sessions as you want, they are only a "virtual" construct.</li> 238 <li><b>Advanced: Don't use <tt>Session.execCommand()</tt>, but rather aquire a shell with 239 <tt>Session.startShell()</tt></b>. See also <a href="#env">this question</a>.</li> 240 </ul> 241 </p> 242 243 244 [<a href="#oben">TOP</a>] 245 246 <hr><a name="sessionlimit"></a><h2>I cannot open more than 10 concurrent sessions (or SCP clients).</h2> 247 <p> 248 You are probably using OpenSSH. By looking at their source code you will find out that there 249 is a hard-coded constant called MAX_SESSIONS in the session.c file which is set to "10" by default. 250 This is a per connection limit. Unfortunately, it is not a run-time tunable parameter. 251 However, this limit has no effect on the number of concurrent port forwardings. Please note: this information 252 is based on the OpenSSH 4.3 release. 253 </p> 254 <p> 255 Possible solutions: 256 <ul> 257 <li>(a) Recompile your SSH daemon</li> 258 <li>(b) Try to live with this limit and keep the number of concurrent sessions <= 10.</li> 259 <li>(c) Distribute your sessions over multiple concurrent SSH connections.</li> 260 </ul> 261 </p> 262 <p> 263 Just for completeness: starting from release 210, the thrown exception may look as follows:<br> 264 <tt> 265 <pre> 266 java.io.IOException: Could not open channel (The server refused to open the channel (SSH_OPEN_ADMINISTRATIVELY_PROHIBITED, 'open failed')) 267 </pre> 268 </tt> 269 </p> 270 271 [<a href="#oben">TOP</a>] 272 273 <hr><a name="passwordauth"></a><h2>Password authentication fails, I get "Authentication method password 274 not supported by the server at this stage".</h2> 275 276 <p> 277 Many default SSH server installations are configured to refuse the authentication type "password". 278 Often, they only accept "publickey" and "keyboard-interactive". You have different options: 279 </p> 280 281 <p> 282 <ul> 283 <li><b>Enable password authentication.</b> E.g., in case of OpenSSH on Fedora, edit 284 <code>/etc/sshd/sshd_config</code> and change the value of "PasswordAuthentication" to "yes", 285 then send a HUP signal to the daemon so that it re-reads its configuration.</li> 286 <li><b>Switch to public-key authentication.</b> Probably the best choice.</li> 287 <li><b>Try to use keyboard-interactive authentication.</b> If you have a GUI that interacts with a user, 288 then this is doable (check out the SwingShell.java example).</li> 289 </ul> 290 </p> 291 292 <p> 293 In general it is a good idea to call either <code>Connection.getRemainingAuthMethods()</code> 294 or <code>Connection.isAuthMethodAvailable()</code> before using a certain authentication method. 295 </p> 296 297 <p> 298 Please note that most servers let you in after one successful authentication step. However, in rare cases 299 you may encounter servers that need several steps. I.e., if one of the <code>Connection.authenticateWithXXX()</code> 300 methods returns <code>false</code> and <code>Connection.isAuthenticationPartialSuccess()</code> returns 301 <code>true</code>, then further authentication is needed. For each step, to find out which authentication methods 302 may proceed, you can use either the <code>Connection.getRemainingAuthMethods()</code> 303 or the <code>Connection.isAuthMethodAvailable()</code> method. Again, please have a look into the 304 SwingShell.java example. 305 </p> 306 307 [<a href="#oben">TOP</a>] 308 309 <hr><a name="puttygen"></a><h2>Why does public key authentication fail with my putty key?</h2> 310 <p> 311 When using putty private keys (e.g., .ppk files) with public key authentication, you get a 312 "Publickey authentication failed" exception. The reason is that the library currently is not able to 313 directly handle private keys in the proprietary format used by putty. However, you can use the 314 "puttygen" tool (from the putty website) to convert your key to the desired format: load your key, 315 then go to the conversions menu and select "Save OpenSSH key" (which saves the key in openssl PEM format, 316 e.g., call it "private.pem"). 317 </p> 318 319 [<a href="#oben">TOP</a>] 320 321 <hr><a name="catmethod"></a><h2>I am sending data to a remote file using the "cat" method, but not all data is being written.</h2> 322 <p> 323 Please read carefully the answer to the following <a href="#pumptoremote">question</a>. 324 </p> 325 326 [<a href="#oben">TOP</a>] 327 328 329 <hr><a name="pumptoremote"></a><h2>I want to pump data into a remote file, but the amount of data to be sent 330 is not known at the time the transfer starts.</h2> 331 <p> 332 The SCP protocol communicates the amount of data to be sent at the start of the transfer, 333 so SCP remains out of consideration. Possible other solutions: 334 <ul> 335 <li>Use the SFTP client. Recommended.</li> 336 <li>Execute "cat > filename.txt" on the remote side and pump the data into stdin. This method is NOT recommended (and won't work on Windows...).</li> 337 </ul> 338 </p> 339 <p> 340 Be careful if you use the "cat" approach, as it may happen that not all your data will be 341 written. If you close the stdin stream and immediatelly close the session (or the whole connection) then 342 some SSH servers do not send the pending data to the process being executed ("cat" in this case). 343 You have to wait until "cat" has received the EOF and terminates before closing the session. However, 344 waiting for the termination may not always work, since SSH servers sometimes "forget" to send the exit code 345 of the remote process. The following code MAY work: 346 </p> 347 <p> 348 <tt> 349 <pre> 350 Session sess = conn.openSession(); 351 sess.execCommand("cat > test.txt"); 352 OutputStream stdin = sess.getStdin(); 353 354 ... out.write(...) ... out.write(...) ... 355 356 /* The following flush() is only needed if you wrap the */ 357 /* stdin stream (e.g., with a BufferedOutputStream). */ 358 out.flush(); 359 360 /* Now let's send EOF */ 361 out.close(); 362 363 /* Let's wait until cat has finished */ 364 sess.waitForCondition(ChannelCondition.EXIT_STATUS, 2000); 365 /* Better: put the above statement into a while loop! */ 366 /* In ANY CASE: read the Javadocs for waitForCondition() */ 367 368 /* Show exit status, if available (otherwise "null") */ 369 System.out.println("ExitCode: " + sess.getExitStatus()); 370 /* Now its hopefully safe to close the session */ 371 sess.close(); 372 </pre> 373 </tt> 374 </p> 375 <p> 376 (Just a thought for another solution: execute <code>cat > test.txt && echo "FINISHED"</code> 377 and wait until you get "FINISHED" on stdout... - try it on your own risk =) 378 </p> 379 380 [<a href="#oben">TOP</a>] 381 382 <hr><a name="swingshell"></a><h2>Do you have an example for the usage of feature XYZ?</h2> 383 <p> 384 Please have at look at the examples section in the distribution, especially at the SwingShell.java example. 385 </p> 386 387 [<a href="#oben">TOP</a>] 388 389 <hr><a name="maven"></a><h2>Where is the official Maven repository?</h2> 390 <p> 391 We regulary get requests for a Maven repository. Please note that <b>there is no such thing as an official Ganymed SSH-2 for Java Maven repository</b>. 392 At the moment, we do not have the resources to support specific build systems (be it Maven or anything else). We know that others 393 have setup (and not maintained) such repositories. However, we believe that you should 394 download security related software only from a trusted source - in other words, download the precompiled .jar file from our website and add it to your 395 project. This generic approach will work with every java development enviroment and build system. 396 Last warning: please think twice before you use a foreign "repository" to "auto-update" security related components of your project. 397 </p> 398 399 [<a href="#oben">TOP</a>] 400 401 </body> 402 </html> 403 404
