additional kex algorithms #60

mpet · 2020-11-17T10:22:58Z

Hi
This again add support for kex algorithms:

ecdh-sha2-nistp256
ecdh-sha2-nistp384
ecdh-sha2-nistp521
curve25519-sha256

I have tested all but curve25519-sha256 vs Mina.

I tried to keep it as small as possible. Changes taken from connectbot/sshlib.
I also added some more tests from the same project.

@kuisathaverat I would appreciate if you could do a new check of code.

//mike

…to minimal_kex

jvz

This PR looks good to me from a code standpoint. I haven't manually tested this, however.

src/com/trilead/ssh2/crypto/dh/DhExchange.java

src/com/trilead/ssh2/crypto/dh/EcDhExchange.java

src/com/trilead/ssh2/crypto/dh/DhExchange.java

src/com/trilead/ssh2/crypto/dh/DhGroupExchange.java

src/com/trilead/ssh2/signature/ECDSAKeyAlgorithm.java

src/com/trilead/ssh2/transport/KexManager.java

mpet · 2020-11-18T12:58:54Z

@jvz

rfc8332 enabled

ssh-dss used for Mina in KexManager.verifySignature()

rsa-sha2-256 used for libssh in KexManager.verifySignature()

Caused by: java.io.IOException: Peer sent wrong signature format
	at com.trilead.ssh2.signature.RSAKeyAlgorithm.decodeSignature(RSAKeyAlgorithm.java:71)
	at com.trilead.ssh2.transport.KexManager.verifySignature(KexManager.java:409)
	at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:671)
	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)

Public key

Sun RSA public key, 2048 bits
  modulus: 24247994701388530985777874400375834166851168539718260762048203164605437776096320696475128500914043328563334993089615828868976641819889953559375831203517411728220021554958175578958518511241198590332408332822434908128048995032985247538084278757682407106185175350494603932101978596444154007217201318721448069013773378485483977957797219137788870714168001383376978609111302513411200180638471272414779931090516432769864984381287463266050292206124775831716656643873116427835139480465883968282667070080850542771363566882056116655280129893585903557956453119780352142516195528623182365943823079911520036863689881913648549544089
  public exponent: 65537

rfc8332 disabled

ssh-dss used for Mina im KexManager.verifySignature()

ssh-rsa used for libssh in in KexManager.verifySignature() then works ok but tests are failing.

@kuisathaverat I saw you had reverted this before. Did you experience something similar? Solved and how?

jvz · 2020-11-18T16:53:07Z

Ideally, we'd like to both support RSA SHA-2 as well as deprecating or removing RSA SHA-1 as that's what OpenSSH has been recommending for several months (possibly a couple years by now) as they'll be removing support in a future release (it's currently disabled by default at least). I believe the code added here about extension negotiation may be necessary to ensure the SSH server supports RSA SHA-2 as my previous attempt at supporting this would simply send the SHA-2 signature first and then retry with the SHA-1 signature if it failed, though some SSH servers may be overly paranoid and reject the second attempt (it is allowed by the RFC after all, though it's a shitty response).

kuisathaverat · 2020-11-18T16:53:56Z

I saw you had reverted this before. Did you experience something similar? Solved and how?

yep, every time we add a new algorithm, those tests have a list of algorithms in order, it should be the same that is returned by the method getPreferredServerHostkeyAlgorithmOrder, and the only way to have the list is to inspect the value is running the tests in debug mode.

kuisathaverat · 2020-11-18T17:16:22Z

in the meantime, I am checking a prototype of trilead-api-plugin and ssh-slaves-plugin, I have to check why the keys encrypted are not working but all the rest is working.

The only key type does not wor is the RSA (old format ssh-rsa)

mpet · 2020-11-19T07:02:40Z

@jvz I can try to add extension negotiation if we cannot live with this commit as is.
Which server did you test on ? I only know Apache Mina that has support for https://tools.ietf.org/html/rfc8332
We really need this commit for our testing of libssh server.

@kuisathaverat how shall we proceed? I need 3 algorithms for testing of libssh server.

mpet · 2020-11-19T09:55:38Z

@jvz I checked and it seems to be quite some changes in to get sha 2 working but I will give it a try.

mpet · 2020-11-19T14:18:46Z

@jvz I checked code and I cannot see that ext info is used for providing info for SHA2. I can see this for HostKey verification.

if (RSASHA1Verify.ID_SSH_RSA.equals(serverHostKeyAlgorithm) ||
			RSASHA256Verify.ID_RSA_SHA_2_256.equals(serverHostKeyAlgorithm) ||
			RSASHA512Verify.ID_RSA_SHA_2_512.equals(serverHostKeyAlgorithm))
		{
			remoteKey = RSASHA1Verify.get().decodePublicKey(serverHostKey);
		}

So it seems SHA1 is used.
I think we need to to a separate submission for sha2. @kuisathaverat what do you think?

jvz · 2020-11-19T14:58:49Z

Oops, sorry, I was referring to the SHA-2 support I had introduced in an earlier commit that we had to revert. I had tested that out with OpenSSH, though we still had regression reports (possible that OpenSSH fixed something itself in newer versions to accept that).

In that code, the RSA public key is still the same regardless if it's using SHA-1 or SHA-2; the hash is only used for host key verification (while signatures are usually used for authentication). So that code snippet is correct at least, @mpet.

mpet · 2020-11-19T17:35:48Z

@jvz so my PR is ok if I fix the test?

jvz · 2020-11-19T18:14:34Z

From my point of view, yes. Ivan is the maintainer, though, so he'll confirm and merge this when he can. I'm on the Jenkins Security team, so I tend to review or audit cryptography and other security changes in Jenkins things, even when we're not the maintainers of the code (it is used in Jenkins after all!).

mpet · 2020-11-19T18:44:15Z

@jvz ok. @kuisathaverat what is your opinion?

mpet · 2020-11-20T06:43:32Z

@kuisathaverat could you make a delivery?

kuisathaverat · 2020-11-20T09:39:50Z

I will test it this weekend, if you need a binary you can use the incremental https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/trilead-ssh2/build-217-jenkins-27-rc201.14ce3b77aeaa/

mpet · 2020-11-20T15:22:26Z

@kuisathaverat thanks! Greatly appreciated.

kuisathaverat · 2020-11-22T20:00:18Z

I've tested that does not break anything, but I did not test if the support for the new kex works, I have to prepare some agents with those kex only enabled.
The pre-release of the trilead-api plugin with these changes can be downloaded from https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/trilead-api/1.0.13-rc47.df7c5d8a2be7/

Along this week I will try to finish my tests.

kuisathaverat · 2020-11-22T20:49:30Z

It was quicker than expected, the curve25519-sha256 seems does not work, it does not appear in the list of algorithms

ssh-agent-curve25519-sha256     | debug1: Forked child 1021.
ssh-agent-curve25519-sha256     | debug1: Set /proc/self/oom_score_adj to 0
ssh-agent-curve25519-sha256     | debug1: rexec start in 5 out 5 newsock 5 pipe 7 sock 8
ssh-agent-curve25519-sha256     | debug1: inetd sockets after dupping: 4, 4
ssh-agent-curve25519-sha256     | Connection from 172.19.0.2 port 41248 on 172.19.0.4 port 22 rdomain ""
ssh-agent-curve25519-sha256     | debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
ssh-agent-curve25519-sha256     | debug1: Remote protocol version 2.0, remote software version TrileadSSH2Java_213
ssh-agent-curve25519-sha256     | debug1: no match: TrileadSSH2Java_213
ssh-agent-curve25519-sha256     | debug1: permanently_set_uid: 105/65534 [preauth]
ssh-agent-curve25519-sha256     | debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
ssh-agent-curve25519-sha256     | debug1: SSH2_MSG_KEXINIT sent [preauth]
ssh-agent-curve25519-sha256     | debug1: SSH2_MSG_KEXINIT received [preauth]
ssh-agent-curve25519-sha256     | debug1: kex: algorithm: (no match) [preauth]
ssh-agent-curve25519-sha256     | Unable to negotiate with 172.19.0.2 port 41248: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521 [preauth]
ssh-agent-curve25519-sha256     | debug1: do_cleanup [preauth]
ssh-agent-curve25519-sha256     | debug1: monitor_read_log: child log fd closed
ssh-agent-curve25519-sha256     | debug1: do_cleanup
ssh-agent-curve25519-sha256     | debug1: Killing privsep child 1022
ssh-agent-curve25519-sha256     | debug1: audit_event: unhandled event 12
ssh-agent-curve25519-sha256     | debug1: main_sigchld_handler: Child exited

This is my test environment https://github.com/kuisathaverat/jenkins-issues/tree/master/kex-algorithms

mpet · 2020-11-23T06:51:10Z

@kuisathaverat I fixed the list issue. Could you have a look again? Also which code formatter do you use for project?

mpet · 2020-11-24T05:26:04Z

@kuisathaverat could you check last commit. Or tell me how to test it?

mpet · 2020-11-25T13:49:51Z

@kuisathaverat could you look into it now?

kuisathaverat · 2020-11-25T14:59:15Z

during the week I do not have too much time, Friday I have time for it. If you want to test it you only have to checkout my test environment and make make start, then you can go to http://localhost:8080, login (admin:password), and check all the agents are connected.

https://github.com/kuisathaverat/jenkins-issues/tree/master/kex-algorithms

mpet · 2020-11-26T11:39:43Z

@kuisathaverat where do I set to use my PR in the testenv?

kuisathaverat · 2020-11-26T11:49:08Z

you have to replace the trilead-api plugin (https://github.com/kuisathaverat/jenkins-issues/tree/master/kex-algorithms/jenkins/jenkins_home/plugins) with the incremental generated by jenkinsci/trilead-api-plugin#20 this file is loaded on you Jenkins instance when you create the Docker container (make clean start)

If you make changes on this PR, a new incremental would be generated, you have to follow the incrementals link in the GitHub checks and take the incremental number

then you have to modify the incremental version and compile again the trilead-api plugin https://github.com/jenkinsci/trilead-api-plugin/pull/20/files

mpet · 2020-11-27T10:11:02Z

@kuisathaverat I failed to do it. I create a PR of jenkinsci/trilead-api-plugin#21 but it fails.
Maybe it is best that you assist me and I can do it next time.

kuisathaverat · 2020-11-28T18:55:32Z

The latest changes fix the issue with the curve25519-sha256 kex, but breaks the rsa-sha2-256 and rsa-sha2-512 key negotiation

ssh-agent-rsa-512               | debug1: inetd sockets after dupping: 4, 4
ssh-agent-rsa-512               | Connection from 192.168.80.3 port 53682 on 192.168.80.8 port 22 rdomain ""
ssh-agent-rsa-512               | debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
ssh-agent-rsa-512               | debug1: Remote protocol version 2.0, remote software version TrileadSSH2Java_213
ssh-agent-rsa-512               | debug1: no match: TrileadSSH2Java_213
ssh-agent-rsa-512               | debug1: permanently_set_uid: 105/65534 [preauth]
ssh-agent-rsa-512               | debug1: list_hostkey_types: rsa-sha2-512 [preauth]
ssh-agent-rsa-512               | debug1: SSH2_MSG_KEXINIT sent [preauth]
ssh-agent-rsa-512               | debug1: SSH2_MSG_KEXINIT received [preauth]
ssh-agent-rsa-512               | debug1: kex: algorithm: diffie-hellman-group-exchange-sha256 [preauth]
ssh-agent-rsa-512               | debug1: kex: host key algorithm: (no match) [preauth]
ssh-agent-rsa-512               | Unable to negotiate with 192.168.80.3 port 53682: no matching host key type found. Their offer: ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ssh-rsa,ssh-dss [preauth]
ssh-agent-rsa-512               | debug1: do_cleanup [preauth]
ssh-agent-rsa-512               | debug1: monitor_read_log: child log fd closed
ssh-agent-rsa-512               | debug1: do_cleanup
ssh-agent-rsa-512               | debug1: Killing privsep child 90
ssh-agent-rsa-512               | debug1: audit_event: unhandled event 12
ssh-agent-rsa-512               | debug1: main_sigchld_handler: Child exited

ssh-agent-rsa-256               | debug1: inetd sockets after dupping: 4, 4
ssh-agent-rsa-256               | Connection from 192.168.80.3 port 59600 on 192.168.80.4 port 22 rdomain ""
ssh-agent-rsa-256               | debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
ssh-agent-rsa-256               | debug1: Remote protocol version 2.0, remote software version TrileadSSH2Java_213
ssh-agent-rsa-256               | debug1: no match: TrileadSSH2Java_213
ssh-agent-rsa-256               | debug1: permanently_set_uid: 105/65534 [preauth]
ssh-agent-rsa-256               | debug1: list_hostkey_types: rsa-sha2-256 [preauth]
ssh-agent-rsa-256               | debug1: SSH2_MSG_KEXINIT sent [preauth]
ssh-agent-rsa-256               | debug1: SSH2_MSG_KEXINIT received [preauth]
ssh-agent-rsa-256               | debug1: kex: algorithm: diffie-hellman-group-exchange-sha256 [preauth]
ssh-agent-rsa-256               | debug1: kex: host key algorithm: (no match) [preauth]
ssh-agent-rsa-256               | Unable to negotiate with 192.168.80.3 port 59600: no matching host key type found. Their offer: ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ssh-rsa,ssh-dss [preauth]
ssh-agent-rsa-256               | debug1: do_cleanup [preauth]
ssh-agent-rsa-256               | debug1: monitor_read_log: child log fd closed
ssh-agent-rsa-256               | debug1: do_cleanup
ssh-agent-rsa-256               | debug1: Killing privsep child 96
ssh-agent-rsa-256               | debug1: audit_event: unhandled event 12
ssh-agent-rsa-256               | debug1: main_sigchld_handler: Child exited

Elisedlund-ericsson · 2020-11-30T07:35:55Z

but breaks the rsa-sha2-256 and rsa-sha2-512 key negotiation

why would they work? (have they ever worked?)
Latest trilead release supports: [ssh-ed25519, ecdsa-sha2-nistp521, ecdsa-sha2-nistp384, ecdsa-sha2-nistp256, ssh-rsa, ssh-dss]
same as "Their offer: ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ssh-rsa,ssh-dss"

This pullrequest is to add kex algorithms and does not state that it adds any host key algorithms.

src/com/trilead/ssh2/signature/KeyAlgorithmManager.java

mpet · 2020-12-01T07:50:34Z

@jvz I have tried rsa-sha2-512 for sshlib 0.9.5 as peer. This is what happens:

In verifySignature() we can see that:

algorithm.getKeyFormat().equals(kxs.np.server_host_key_algo)

kxs.np.server_host_key_algo = rsa-sha2-512

PublicKey publicKey = algorithm.decodePublicKey(hostkey);

The PublicKey is:
Sun RSA public key, 2048 bits
OID: 1.2.840.113549.1.1.1

We try to decode the signature from peer.

byte[] signature = algorithm.decodeSignature(sig);

RSAKeyAlgorithm class we do decodeSignature:

 if (!sig_format.equals(getKeyFormat())) {

sig_format is 'ssh-rsa' but should be 'rsa-sha2-512'

So this means the signature for peer is not rsa-sha-512

So the signature in response from peer:

PacketKexDHReply dhr = new PacketKexDHReply(msg, 0, msglen);

contains a signature using 'ssh-rsa'

How is it intended to work?

mpet · 2020-12-01T13:03:13Z

@kuisathaverat could you make yet another check? i got it working with our product under test.

jvz · 2020-12-01T14:59:09Z

Ha, I love this RFC because you, just like me, have been confused by what exactly the damn RFC is modifying. The key format is still called "ssh-rsa" because it's just the RSA key; the only change here is for signatures which follow the sha2 format. This same problem has popped up in other SSH libraries during development of the same feature. An ssh-rsa signature is still sha1, but an ssh-rsa host key or authentication key will remain the same for sha2.

jvz · 2020-12-01T15:34:10Z

Relevant RFC: https://tools.ietf.org/html/rfc8332

Note that there might be a bug (or missing feature?) with some SSH implementations related to trying to use the SHA-2 variant first before falling back to trying again with SHA-1. If there's some extension negotiation info that can help reduce the unnecessary encoding attempts, that might be what you're seeing. It's been a while since I was deep in that code (the latest SSH-related changes I've worked on elsewhere (apache/mina-sshd#177) were related to curve25519-sha256 and curve448-sha512 support similar to this PR).

A potentially related commit in Mina that might help you figure this out:

apache/mina-sshd@b9ddc14

mpet · 2020-12-03T06:20:04Z

@kuisathaverat any time to check again? would be nice with a README on how to test :-)

mpet · 2020-12-06T09:13:23Z

@kuisathaverat this is a friendly bump about running tests to verify ;-)

jvz · 2020-12-07T14:57:00Z

Thanks for handling this, Ivan!

In my company we use trilead with protobuf-java excluded and it been working fine, (we use all of the KEX algorithms jenkinsci#60 which is the reason why tink was added and indirectly protobuf-java )

In my company we use trilead with protobuf-java excluded and it been working fine, (we use all of the KEX algorithms #60 which is the reason why tink was added and indirectly protobuf-java )

eraonel added 3 commits November 17, 2020 11:13

additional kex algorithms

1226777

additional kex algorithms

9eb96da

Merge branch 'minimal_kex' of https://github.com/mpet/trilead-ssh2 in…

c03eb37

…to minimal_kex

jvz approved these changes Nov 17, 2020

View reviewed changes

src/com/trilead/ssh2/crypto/dh/DhExchange.java Outdated Show resolved Hide resolved

src/com/trilead/ssh2/crypto/dh/EcDhExchange.java Outdated Show resolved Hide resolved