-
-
Notifications
You must be signed in to change notification settings - Fork 8.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[JENKINS-41745] Non-Remoting-based CLI #2795
Merged
Merged
Changes from 48 commits
Commits
Show all changes
64 commits
Select commit
Hold shift + click to select a range
492dbbe
[JENKINS-41745] Make jenkins-cli.jar connect to the SSH port by default.
jglick 2fe2487
FindBugs, and more clearly stating which transport is in use.
jglick 8302b85
Draft of a new CLI transport that can operate over CLIAction with the…
jglick 7da0d39
Argument processing mistake.
jglick 1e860c8
Apply SUREFIRE-1226 workaround across all modules, including cli.
jglick 624ba59
Allow tests to run which use CLI.<init> in process.
jglick 4df2bf1
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick c2a5d85
Establishing baseline behavior of JENKINS-12543: no workaround when u…
jglick 8d622e0
Clarifying role of API tokens in -remoting.
jglick 12ae48e
Deprecating --username/--password and login/logout in favor of new -a…
jglick b8ad360
Marking various APIs, and a couple of commands, deprecated when they …
jglick 82cd1bd
Deleteing apparently unused class SequenceOutputStream.
jglick 5f0bb2a
Some more work deprecating channel/checkChannel.
jglick 9ffb5d8
Hoping to fix an unreproducible test hang.
jglick 2cf0ac5
Pick up https://github.com/jenkinsci/sshd-module/pull/10 so we are us…
jglick 9e427ce
Noting potential issue in ConsoleCommand.
jglick f437ccc
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick 9434d0e
UI to enable or disable CLI over Remoting.
jglick 71db95a
Allow http://repo.jenkins-ci.org/public/org/jenkins-ci/modules/sshd/1…
jglick 996c52e
Trying to avoid a premature test timeout.
jglick 59d5013
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick 473744c
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick 289db88
Using Logger consistently for all messages that are not expected to b…
jglick f3dae19
Fixed locale handling.
jglick 7174e3e
Allowing install-plugin to load a file from standard input so as to w…
jglick 7baf81d
Allowing `build -p fileParam= prj` to load a file from standard input…
jglick e69ba0f
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick 60632c0
Added -strictHostKey option to CLI in -ssh mode.
jglick 466d74d
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick f168936
Test failure on Windows, perhaps specific to filesystem type of Tempo…
jglick 551808c
Using kill -QUIT to try to diagnose client hangs observed on ci.jenki…
jglick f6314b7
Testing interrupt behavior.
jglick 9812db7
Improved handling of stream closure.
jglick 71d7562
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick 56cee95
Found another case where the Windows CI build was complaining about k…
jglick c74999f
Found a race condition which could explain random CI hangs of CLIActi…
jglick 4c1910a
Silence an annoying message.
jglick 4069e67
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick 03aa2a1
Simpler to catch ReadPendingException in just one place.
jglick fb311c6
Suppress a meaningless ClosedChannelException thrown by Jetty when ki…
jglick caf3e8e
Removing comment corresponding to something I can no longer reproduce.
jglick 7f633de
CLITest.interrupt could fail if you had a stale localhost entry in ~/…
jglick f7006b4
Jetty 9.4.3.v20170317 refuses to set an empty HTTP header, leading to…
jglick 3b92399
Tested Jetty 9.4.3.v20170317 but ReadPendingException’s did not disap…
jglick 94d5f96
Figured out why Security232Test.commonsCollections1 was being skipped…
jglick dbe24a8
Using CLICommandInvoker in cases where we did not actually need to be…
jglick 5caee58
Noting that ssh-cli-auth is obsolete.
jglick 9fc59bf
Finally figured out how to suppress the ReadPendingException from Jetty.
jglick 7bca0a1
Refined implementation of -logger.
jglick 3939820
Using logger as suggested by @rsandell.
jglick da58159
-http mode broke when the user omitted the mandatory final `/` in the…
jglick 724ebbc
Improved error message displayed when using -http against a pre-JENKI…
jglick 49f0360
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick c41c97b
@stephenc requested documentation of the deprecated FullDuplexHttpStr…
jglick 6c63890
Demonstrating that interleaved stdio does work in -http mode.
jglick 9150d5b
Suppress an idle timeout from Jetty which would otherwise interfere w…
jglick 6ffc57a
Verifying that PlainCLIProtocol ignores unrecognized opcodes, so long…
jglick 2e7fe04
Merge branch 'master' into SSH-CLI-JENKINS-41745
jglick f3da0e4
Minor review comments from @oleg-nenashev.
jglick 0f87e10
@oleg-nenashev requested this be decoupled from https://github.com/je…
jglick 98f227c
Showing general help message rather than suggesting obsolete authenti…
jglick 45092f0
checkChannel() provides a better error message in case you are not us…
jglick 27d508d
sshd 1.11
jglick 7ae404c
Revert "Noting that ssh-cli-auth is obsolete."
jglick File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
55 changes: 55 additions & 0 deletions
55
cli/src/main/java/hudson/cli/DiagnosedStreamCorruptionException.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
package hudson.cli; | ||
|
||
import java.io.PrintWriter; | ||
import java.io.StreamCorruptedException; | ||
import java.io.StringWriter; | ||
|
||
// TODO COPIED FROM hudson.remoting | ||
|
||
/** | ||
* Signals a {@link StreamCorruptedException} with some additional diagnostic information. | ||
* | ||
* @author Kohsuke Kawaguchi | ||
*/ | ||
class DiagnosedStreamCorruptionException extends StreamCorruptedException { | ||
private final Exception diagnoseFailure; | ||
private final byte[] readBack; | ||
private final byte[] readAhead; | ||
|
||
DiagnosedStreamCorruptionException(Exception cause, Exception diagnoseFailure, byte[] readBack, byte[] readAhead) { | ||
initCause(cause); | ||
this.diagnoseFailure = diagnoseFailure; | ||
this.readBack = readBack; | ||
this.readAhead = readAhead; | ||
} | ||
|
||
public Exception getDiagnoseFailure() { | ||
return diagnoseFailure; | ||
} | ||
|
||
public byte[] getReadBack() { | ||
return readBack; | ||
} | ||
|
||
public byte[] getReadAhead() { | ||
return readAhead; | ||
} | ||
|
||
@Override | ||
public String toString() { | ||
StringBuilder buf = new StringBuilder(); | ||
buf.append(super.toString()).append("\n"); | ||
buf.append("Read back: ").append(HexDump.toHex(readBack)).append('\n'); | ||
buf.append("Read ahead: ").append(HexDump.toHex(readAhead)); | ||
if (diagnoseFailure!=null) { | ||
StringWriter w = new StringWriter(); | ||
PrintWriter p = new PrintWriter(w); | ||
diagnoseFailure.printStackTrace(p); | ||
p.flush(); | ||
|
||
buf.append("\nDiagnosis problem:\n "); | ||
buf.append(w.toString().trim().replace("\n","\n ")); | ||
} | ||
return buf.toString(); | ||
} | ||
} |
191 changes: 191 additions & 0 deletions
191
cli/src/main/java/hudson/cli/FlightRecorderInputStream.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,191 @@ | ||
package hudson.cli; | ||
|
||
import java.io.ByteArrayOutputStream; | ||
import java.io.IOException; | ||
import java.io.InputStream; | ||
import java.io.OutputStream; | ||
import java.util.Arrays; | ||
|
||
// TODO COPIED FROM hudson.remoting | ||
|
||
/** | ||
* Filter input stream that records the content as it's read, so that it can be reported | ||
* in case of a catastrophic stream corruption problem. | ||
* | ||
* @author Kohsuke Kawaguchi | ||
*/ | ||
class FlightRecorderInputStream extends InputStream { | ||
|
||
/** | ||
* Size (in bytes) of the flight recorder ring buffer used for debugging remoting issues. | ||
* @since 2.41 | ||
*/ | ||
static final int BUFFER_SIZE = Integer.getInteger("hudson.remoting.FlightRecorderInputStream.BUFFER_SIZE", 1024 * 1024); | ||
|
||
private final InputStream source; | ||
private ByteArrayRingBuffer recorder = new ByteArrayRingBuffer(BUFFER_SIZE); | ||
|
||
FlightRecorderInputStream(InputStream source) { | ||
this.source = source; | ||
} | ||
|
||
/** | ||
* Rewinds the record buffer and forget everything that was recorded. | ||
*/ | ||
public void clear() { | ||
recorder = new ByteArrayRingBuffer(BUFFER_SIZE); | ||
} | ||
|
||
/** | ||
* Gets the recorded content. | ||
*/ | ||
public byte[] getRecord() { | ||
return recorder.toByteArray(); | ||
} | ||
|
||
/** | ||
* Creates a {@link DiagnosedStreamCorruptionException} based on the recorded content plus read ahead. | ||
* The caller is responsible for throwing the exception. | ||
*/ | ||
public DiagnosedStreamCorruptionException analyzeCrash(Exception problem, String diagnosisName) { | ||
final ByteArrayOutputStream readAhead = new ByteArrayOutputStream(); | ||
final IOException[] error = new IOException[1]; | ||
|
||
Thread diagnosisThread = new Thread(diagnosisName+" stream corruption diagnosis thread") { | ||
public void run() { | ||
int b; | ||
try { | ||
// not all InputStream will look for the thread interrupt flag, so check that explicitly to be defensive | ||
while (!Thread.interrupted() && (b=source.read())!=-1) { | ||
readAhead.write(b); | ||
} | ||
} catch (IOException e) { | ||
error[0] = e; | ||
} | ||
} | ||
}; | ||
|
||
// wait up to 1 sec to grab as much data as possible | ||
diagnosisThread.start(); | ||
try { | ||
diagnosisThread.join(1000); | ||
} catch (InterruptedException ignored) { | ||
// we are only waiting for a fixed amount of time, so we'll pretend like we were in a busy loop | ||
Thread.currentThread().interrupt(); | ||
// fall through | ||
} | ||
|
||
IOException diagnosisProblem = error[0]; // capture the error, if any, before we kill the thread | ||
if (diagnosisThread.isAlive()) | ||
diagnosisThread.interrupt(); // if it's not dead, kill | ||
|
||
return new DiagnosedStreamCorruptionException(problem,diagnosisProblem,getRecord(),readAhead.toByteArray()); | ||
|
||
} | ||
|
||
@Override | ||
public int read() throws IOException { | ||
int i = source.read(); | ||
if (i>=0) | ||
recorder.write(i); | ||
return i; | ||
} | ||
|
||
@Override | ||
public int read(byte[] b, int off, int len) throws IOException { | ||
len = source.read(b, off, len); | ||
if (len>0) | ||
recorder.write(b,off,len); | ||
return len; | ||
} | ||
|
||
/** | ||
* To record the bytes we've skipped, convert the call to read. | ||
*/ | ||
@Override | ||
public long skip(long n) throws IOException { | ||
byte[] buf = new byte[(int)Math.min(n,64*1024)]; | ||
return read(buf,0,buf.length); | ||
} | ||
|
||
@Override | ||
public int available() throws IOException { | ||
return source.available(); | ||
} | ||
|
||
@Override | ||
public void close() throws IOException { | ||
source.close(); | ||
} | ||
|
||
@Override | ||
public boolean markSupported() { | ||
return false; | ||
} | ||
|
||
// http://stackoverflow.com/a/3651696/12916 | ||
private static class ByteArrayRingBuffer extends OutputStream { | ||
|
||
byte[] data; | ||
|
||
int capacity, pos = 0; | ||
|
||
boolean filled = false; | ||
|
||
public ByteArrayRingBuffer(int capacity) { | ||
data = new byte[capacity]; | ||
this.capacity = capacity; | ||
} | ||
|
||
@Override | ||
public synchronized void write(int b) { | ||
if (pos == capacity) { | ||
filled = true; | ||
pos = 0; | ||
} | ||
data[pos++] = (byte) b; | ||
} | ||
|
||
public synchronized byte[] toByteArray() { | ||
if (!filled) { | ||
return Arrays.copyOf(data, pos); | ||
} | ||
byte[] ret = new byte[capacity]; | ||
System.arraycopy(data, pos, ret, 0, capacity - pos); | ||
System.arraycopy(data, 0, ret, capacity - pos, pos); | ||
return ret; | ||
} | ||
|
||
/** @author @roadrunner2 */ | ||
@Override public synchronized void write(byte[] buf, int off, int len) { | ||
// no point in trying to copy more than capacity; this also simplifies logic below | ||
if (len > capacity) { | ||
off += (len - capacity); | ||
len = capacity; | ||
} | ||
|
||
// copy to buffer, but no farther than the end | ||
int num = Math.min(len, capacity - pos); | ||
if (num > 0) { | ||
System.arraycopy(buf, off, data, pos, num); | ||
off += num; | ||
len -= num; | ||
pos += num; | ||
} | ||
|
||
// wrap around if necessary | ||
if (pos == capacity) { | ||
filled = true; | ||
pos = 0; | ||
} | ||
|
||
// copy anything still left | ||
if (len > 0) { | ||
System.arraycopy(buf, off, data, pos, len); | ||
pos += len; | ||
} | ||
} | ||
|
||
} | ||
|
||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
package hudson.cli; | ||
|
||
// TODO COPIED FROM hudson.remoting | ||
|
||
/** | ||
* @author Kohsuke Kawaguchi | ||
*/ | ||
class HexDump { | ||
private static final String CODE = "0123456789abcdef"; | ||
|
||
public static String toHex(byte[] buf) { | ||
return toHex(buf,0,buf.length); | ||
} | ||
public static String toHex(byte[] buf, int start, int len) { | ||
StringBuilder r = new StringBuilder(len*2); | ||
boolean inText = false; | ||
for (int i=0; i<len; i++) { | ||
byte b = buf[start+i]; | ||
if (b >= 0x20 && b <= 0x7e) { | ||
if (!inText) { | ||
inText = true; | ||
r.append('\''); | ||
} | ||
r.append((char) b); | ||
} else { | ||
if (inText) { | ||
r.append("' "); | ||
inText = false; | ||
} | ||
r.append("0x"); | ||
r.append(CODE.charAt((b>>4)&15)); | ||
r.append(CODE.charAt(b&15)); | ||
if (i < len - 1) { | ||
if (b == 10) { | ||
r.append('\n'); | ||
} else { | ||
r.append(' '); | ||
} | ||
} | ||
} | ||
} | ||
if (inText) { | ||
r.append('\''); | ||
} | ||
return r.toString(); | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's already in the core due to the sshd-module
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, it is in
war
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, nvm