[JENKINS-41745] Non-Remoting-based CLI #2795

jglick · 2017-03-10T21:41:35Z

Downstream of jenkinsci/sshd-plugin#12. Also cf. jenkinsci/support-core-plugin#110, jenkins-infra/jenkins.io#802, jenkinsci-cert/SECURITY-218#8, #2834.

Proposed changelog entries:

JENKINS-41745: the use of Jenkins CLI in its traditional “Remoting” mode has been deprecated for security reasons; this mode will be disabled on new installations, and new downloads of jenkins-cli.jar require a -remoting option to use it. You may continue to access the CLI using a stock ssh command; or you may download a new jenkins-cli.jar supporting a new, default HTTP-based protocol as well as SSH-based access. See the client help text for options, as well as the CLI documentation.
The install-plugin command now supports a source of = to load a plugin *.hpi from standard input, since use of local file paths only worked in Remoting mode.
The build command’s -p option applied to a file parameter now supports a blank filename to load contents from standard input, since use of local file paths only worked in Remoting mode.
JENKINS-33595: the SSH port is now disabled by default on new installations. Use Configure Global Security to enable it.

@reviewbybees & @jenkinsci/code-reviewers

… regular HTTP(S) port.

@argument

…sing Remoting transport other than SSH authentication. (Verifying that this affects only @argument in CLICommand, not @CLIMethod.) With the new HTTP protocol in JENKINS-41745, API tokens may be used to set a transport authentication.

…uth option passing BASIC authentication to /cli endpoint. Simpler, does not rely on Remoting, allows use of API tokens, and bypasses JENKINS-12543. (You could actually do this before but only by embedding userinfo in the -s URL, especially awkward for usernames containing @.)

…assume a Remoting-based CLI.

… of sshd-core for client & server.

….11-SNAPSHOT/sshd-1.11-20170315.153852-1.jar to be downloaded.

…e stdout/stderr.

…ork without -remoting.

.

…cation options.

…ing Remoting mode.

oleg-nenashev

sshConnection() uses Future await() calls without timeouts. There is also a case when the ClientSession may be leaked (didn't dig into).

oleg-nenashev · 2017-04-07T16:45:03Z

cli/src/main/java/hudson/cli/SSHCLI.java

+            client.setServerKeyVerifier(verifier);
+            client.start();
+
+            ConnectFuture cf = client.connect(user, sshHost, sshPort);


client.stop() won't be invoked if this method or await() fail. Not sure if it matters

Should not matter. The process will exit anyway.

oleg-nenashev · 2017-04-07T16:45:33Z

cli/src/main/java/hudson/cli/SSHCLI.java

+            client.start();
+
+            ConnectFuture cf = client.connect(user, sshHost, sshPort);
+            cf.await();


🐛 no timeout

So? Either it connects eventually; or the socket gives a timeout, in which case we get an exception; or the socket does not give a timeout, in which case the client waits and the user eventually kills it. Why would we need to impose an additional timeout?

in which case the client waits and the user eventually kills it

Jenkins CLI is also a tool for automation. Such possible hanging is not a good UX IMHO. Can convert to Ant if you feel strongly && can handle the possible fallout later if it happens

Jenkins CLI is also a tool for automation.

Sure, and automated scripts typically have their own timeouts.

handle the possible fallout later if it happens

Remember that if anyone is dissatisfied with the quality of implementation of SSHCLI they can simply use an OpenSSH (or whatever) client, with its dozens of options and years of tuning and bug fixes. -ssh mode is provided as a convenience to minimize the friction of switching away from Remoting.

oleg-nenashev · 2017-04-07T16:45:52Z

cli/src/main/java/hudson/cli/SSHCLI.java

+                    channel.setOut(new NoCloseOutputStream(System.out));
+                    channel.setErr(new NoCloseOutputStream(System.err));
+                    WaitableFuture wf = channel.open();
+                    wf.await();


no timeout as well

This reverts commit 5caee58. In fact ssh-cli-auth contains UserPropertyImpl which is used by the sshd module.

oleg-nenashev

OK, here we go 🐝

jglick · 2017-04-07T23:09:08Z

cli/src/main/java/hudson/cli/PlainCLIProtocol.java

+                            handleClose();
+                            break; // TODO verify that we hit EOF immediately, not partway into framelen
+                        } catch (IOException x) {
+                            if (x.getCause() instanceof TimeoutException) {


On Tomcat 8 it seems this is a SocketTimeoutException. Have not yet checked whether catching that (and continuing) allows a long-running groovysh or the like to work.

We could also send empty ping packets in either direction after every 15s or so of inactivity on that side, but there is no way to know what timeouts will be applied by different containers.

JENKINS-46659

oleg-nenashev · 2017-04-08T07:50:12Z

Well, I expected it to stay there for another day after @reviewbybees done and to ping code-reviewers. I am also not a big fan of merging PRs with 64 commits, which do not have JIRA ticket prefix or whatever. It really complicates backporting IMHO.

jglick · 2017-04-10T19:45:03Z

cli/src/main/java/hudson/cli/FullDuplexHttpStream.java

+    }
+
+    /**
+     * @param base the base URL of Jenkins
     * @param target


Oops, should be relativeTarget.

(cherry picked from commit 7290169)

* Do not load detached plugins during tests. Also warn about plugins which force us to create classes.jar. * Porting POM fix from #2795: we must allow snapshots from repo.jenkins-ci.org so as to be able to test upstream component PRs. * Fixed failures in tests which assumed detached plugins were being loaded. * jenkins-test-harness 2.20 * Unreproducible test failure on CI, probably due to timing.

[JENKINS-41745] Make jenkins-cli.jar connect to the SSH port by default.

492dbbe

jglick added the work-in-progress The PR is under active development, not ready to the final review label Mar 10, 2017

jglick added 2 commits March 10, 2017 17:44

FindBugs, and more clearly stating which transport is in use.

2fe2487

Draft of a new CLI transport that can operate over CLIAction with the…

8302b85

… regular HTTP(S) port.

daniel-beck self-requested a review March 11, 2017 22:11

jglick mentioned this pull request Mar 13, 2017

[JENKINS-42191] Enhance CLI HTTP proxy support #2711

Merged

jglick added 11 commits March 13, 2017 11:53

Argument processing mistake.

7da0d39

Apply SUREFIRE-1226 workaround across all modules, including cli.

1e860c8

Allow tests to run which use CLI.<init> in process.

624ba59

Merge branch 'master' into SSH-CLI-JENKINS-41745

4df2bf1

Clarifying role of API tokens in -remoting.

8d622e0

Marking various APIs, and a couple of commands, deprecated when they …

b8ad360

…assume a Remoting-based CLI.

Deleteing apparently unused class SequenceOutputStream.

82cd1bd

Some more work deprecating channel/checkChannel.

5f0bb2a

Hoping to fix an unreproducible test hang.

9ffb5d8

jglick mentioned this pull request Mar 15, 2017

Updating sshd-core to 1.4.0 jenkinsci/sshd-plugin#10

Merged

jglick added 6 commits March 15, 2017 11:51

Pick up jenkinsci/sshd-plugin#10 so we are using a consistent version…

2cf0ac5

… of sshd-core for client & server.

Noting potential issue in ConsoleCommand.

9e427ce

Merge branch 'master' into SSH-CLI-JENKINS-41745

f437ccc

UI to enable or disable CLI over Remoting.

9434d0e

Allow http://repo.jenkins-ci.org/public/org/jenkins-ci/modules/sshd/1…

71db95a

….11-SNAPSHOT/sshd-1.11-20170315.153852-1.jar to be downloaded.

Trying to avoid a premature test timeout.

996c52e

jglick mentioned this pull request Mar 17, 2017

Trying to diagnose a test failure on Windows on CI #2809

Merged

jglick added 5 commits March 17, 2017 10:39

Merge branch 'master' into SSH-CLI-JENKINS-41745

59d5013

Merge branch 'master' into SSH-CLI-JENKINS-41745

473744c

Using Logger consistently for all messages that are not expected to b…

289db88

…e stdout/stderr.

Fixed locale handling.

f3dae19

Allowing install-plugin to load a file from standard input so as to w…

7174e3e

…ork without -remoting.

jglick added 2 commits April 7, 2017 09:56

Merge branch 'master' into SSH-CLI-JENKINS-41745

2e7fe04

Minor review comments from @oleg-nenashev.

f3da0e4

jglick mentioned this pull request Apr 7, 2017

[JENKINS-33595] Disable SSHD port by default on new installations jenkinsci/sshd-plugin#12

Merged

jglick added 4 commits April 7, 2017 11:29

@oleg-nenashev requested this be decoupled from jenkinsci/sshd-plugin#10

0f87e10

.

Showing general help message rather than suggesting obsolete authenti…

98f227c

…cation options.

checkChannel() provides a better error message in case you are not us…

45092f0

…ing Remoting mode.

sshd 1.11

27d508d

oleg-nenashev requested changes Apr 7, 2017

View reviewed changes

Revert "Noting that ssh-cli-auth is obsolete."

7ae404c

This reverts commit 5caee58. In fact ssh-cli-auth contains UserPropertyImpl which is used by the sshd module.

oleg-nenashev approved these changes Apr 7, 2017

View reviewed changes

jglick mentioned this pull request Apr 7, 2017

[JENKINS-16337] Form validation for SSH public keys jenkinsci/ssh-cli-auth-module#4

Merged

jglick added ready-for-merge The PR is ready to go, and it will be merged soon if there is no negative feedback and removed needs-more-reviews Complex change, which would benefit from more eyes labels Apr 7, 2017

jglick commented Apr 7, 2017

View reviewed changes

jglick merged commit 7ae404c into jenkinsci:master Apr 8, 2017

jglick added a commit that referenced this pull request Apr 8, 2017

[FIXED JENKINS-41745] Merged #2795: non-Remoting-based CLI.

7290169

jglick deleted the SSH-CLI-JENKINS-41745 branch April 8, 2017 00:12

jglick commented Apr 10, 2017

View reviewed changes

jglick mentioned this pull request Apr 11, 2017

Added logging to FullDuplexHttpService #2841

Merged

jglick mentioned this pull request Apr 20, 2017

[JENKINS-39738, JENKINS-43668] - Pick up SSHD module including sshd-core 1.6.0 #2853

Merged

kohsuke pushed a commit that referenced this pull request Apr 26, 2017

[FIXED JENKINS-41745] Merged #2795: non-Remoting-based CLI.

de629fc

(cherry picked from commit 7290169)

oc243 mentioned this pull request May 2, 2017

puppet_helper.groovy uses deprecated remoting mode voxpupuli/puppet-jenkins#749

Closed

jglick mentioned this pull request May 4, 2017

[JENKINS-27299] Define disabled in ParameterizedJob rather than AbstractProject #2866

Merged

p0d0l mentioned this pull request May 5, 2017

Add Jenkins 2.54 new cli options: -http, -ssh, -remoting to jenkins_script resource sous-chefs/jenkins#599

Closed

jglick mentioned this pull request Oct 5, 2017

Async startup of the SSHD server jenkinsci/sshd-plugin#19

Merged

jglick mentioned this pull request Jan 9, 2019

Removing code to support the Remoting-based CLI #3838

Merged

jglick mentioned this pull request Jul 19, 2019

Stop using the "name" argument in the "install-plugin" CLI command #4123

Merged

daniel-beck mentioned this pull request Apr 29, 2021

Remove HTTP response headers for the remoting-based CLI #5452

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JENKINS-41745] Non-Remoting-based CLI #2795

[JENKINS-41745] Non-Remoting-based CLI #2795

jglick commented Mar 10, 2017 •

edited

Loading

oleg-nenashev left a comment

oleg-nenashev Apr 7, 2017

jglick Apr 7, 2017

oleg-nenashev Apr 7, 2017

jglick Apr 7, 2017

oleg-nenashev Apr 7, 2017

jglick Apr 7, 2017

oleg-nenashev Apr 7, 2017

jglick Apr 7, 2017

oleg-nenashev left a comment

jglick Apr 7, 2017

jglick Sep 5, 2017

oleg-nenashev commented Apr 8, 2017

jglick Apr 10, 2017

[JENKINS-41745] Non-Remoting-based CLI #2795

[JENKINS-41745] Non-Remoting-based CLI #2795

Conversation

jglick commented Mar 10, 2017 • edited Loading

oleg-nenashev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-nenashev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-nenashev commented Apr 8, 2017

Choose a reason for hiding this comment

jglick commented Mar 10, 2017 •

edited

Loading