Sending data to stdin of remote process - closing has no effect ->remote process waits indefinitely

[jpstotz]

Hi, i am trying to send some data to the stdin of the remote process via sshj.
The sending itself is not a big problem (using Command.getOutputStream() ), however some commands do not terminate until stdin is closed. In seems like it is impossible to use such commands via sshj as closing the OutputStream doe snot has any effect on the remote stdin.

I created the following simple example to demonstarte the problem.

It executes remotely the command cat which simply prints everything on stdout it receives on stdin. Via sshj I am sending the data "Line1\nLine2\nLine3\n" to it and I see that this works as the listening thread prints the received lines correctly.
However in the end this always end in an ConnectionException: Timeout expired as the remote cat process does not terminate because stdin is not closed.

What do I have to send to close the stdin on remote side after the sent data has been received and processed by cat (note this just an example. I want to send larger data to a tar process to extract it on remote side without saving the tar file first)?

	try (Session session = ssh.startSession()) {
		final Command cmd = session.exec("cat");

		Thread t = new Thread() {

			@Override
			public void run() {
				try {
					InputStream in = cmd.getInputStream();
					BufferedReader br = new BufferedReader(new InputStreamReader(in));
					String line;
					while ((line = br.readLine()) != null) {
						System.out.println("## " + line);
					}
				} catch (Exception e) {
					e.printStackTrace();
				}
			}
		};
		t.start();

		byte[] data = "Line1\nLine2\nLine3\n".getBytes();
		OutputStream out = cmd.getOutputStream();
		out.write(data);
		out.flush();
		out.close(); // this should close stdin on remote side but it doesn't do

		// cmd.getInputStream()
		cmd.join(5, TimeUnit.SECONDS);
		System.out.println("\n** exit status: " + cmd.getExitStatus());
	} catch (Exception e) {
		e.printStackTrace();
	} finally {
		ssh.disconnect();
	}

[jpstotz]

Based on the commented out line in ChannelOutputStream.close() I got some new ideas:

https://github.com/hierynomus/sshj/blob/master/src/main/java/net/schmizz/sshj/connection/channel/ChannelOutputStream.java#L166

Git blame has pointed me to #143 in which this line was commented out, reverted, commented out again and so on. In the end #143 has been closed with that line still commented out (what I don't understand as the issue is therefore IMHO not solved).

If i get the discussion of #143 correctly the result was that mostly sending an CHANNEL_EOF is redundant, but if you turn it around this means that is some cases (like the one I describe here) it is not redundant or to be precises it is necessary to send this package!

Therefore I still don't understand why nobody implemented an simple alternative way to send this packet.

[kholoudasem]

Hello @jpstotz ,
Did you manage to find a working solution for:
"send larger data to a tar process to extract it on remote side without saving the tar file first" ? as I'm trying to do the same.

[jpstotz]

@kholoudasem If I remember correctly I tried to optimize a process that uses temporary files on the ssh-server side. As you can read in the linked PR #554 I tried to get the necessary changes into sshj but the whole process stalled. To my feeling hierynomus does not want to touch that code for an unknown reason.

Therefore I have to answer your question with no, I wasn't able to find a solution using a non-modified version of sshj.
Feel free to use the PR or this issue to discuss the topic with hierynomus, may be you have more luck in making him understand why the current sshj code should be improved to allow this use case.

[kholoudasem]

@jpstotz I found one of the workarounds in PR #554 as:
ssh.getTransport().write(new SSHPacket(Message.CHANNEL_EOF).putUInt32(session.getRecipient()));
So that the outstream is closed, but still I don't know how can I use that to tar a large directory and redirect it to remote host without saving it. I posted another PR #719, but no reply yet.

[jpstotz]

Just don't use the -f option of tar. Then the created tar archive is written to stdout and you can receive it on the other side of the ssh session.