new: Support TLS-PSK (TLS 1.3) (#228)

* uTLS: X25519Kyber768Draft00 hybrid post-quantum key agreement by cloudflare/go (#222)

* crypto/tls: Add hybrid post-quantum key agreement  (#13)

* import: client-side KEM from cloudflare/go

* import: server-side KEM from cloudflare/go

* fix: modify test to get rid of CFEvents.

Note: uTLS does not promise any server-side functionality, and this change is made to be able to conduct unit tests which requires both side to be able to handle KEM Curves.

Co-authored-by: Christopher Wood <caw@heapingbits.net>
Co-Authored-By: Bas Westerbaan <bas@westerbaan.name>

----

Based on:

* crypto/tls: Add hybrid post-quantum key agreement 

Adds X25519Kyber512Draft00, X25519Kyber768Draft00, and
P256Kyber768Draft00 hybrid post-quantum key agreements with temporary
group identifiers.

The hybrid post-quantum key exchanges uses plain X{25519,448} instead
of HPKE, which we assume will be more likely to be adopted. The order
is chosen to match CECPQ2.

Not enabled by default.

Adds CFEvents to detect `HelloRetryRequest`s and to signal which
key agreement was used.

Co-authored-by: Christopher Wood <caw@heapingbits.net>

 [bas, 1.20.1: also adds P256Kyber768Draft00]
 [pwu, 1.20.4: updated circl to v1.3.3, moved code to cfevent.go]

* crypto: add support for CIRCL signature schemes

* only partially port the commit from cloudflare/go. We would stick to the official x509 at the cost of incompatibility.

Co-Authored-By: Bas Westerbaan <bas@westerbaan.name>
Co-Authored-By: Christopher Patton <3453007+cjpatton@users.noreply.github.com>
Co-Authored-By: Peter Wu <peter@lekensteyn.nl>

* crypto/tls: add new X25519Kyber768Draft00 code point

Ported from cloudflare/go to support the upcoming new post-quantum keyshare.

----

* Point tls.X25519Kyber768Draft00 to the new 0x6399 identifier while the
  old 0xfe31 identifier is available as tls.X25519Kyber768Draft00Old.
* Make sure that the kem.PrivateKey can always be mapped to the CurveID
  that was linked to it. This is needed since we now have two ID
  aliasing to the same scheme, and clients need to be able to detect
  whether the key share presented by the server actually matches the key
  share that the client originally sent.
* Update tests, add the new identifier and remove unnecessary code.

Link: https://mailarchive.ietf.org/arch/msg/tls/HAWpNpgptl--UZNSYuvsjB-Pc2k/
Link: https://datatracker.ietf.org/doc/draft-tls-westerbaan-xyber768d00/02/
Co-Authored-By: Peter Wu <peter@lekensteyn.nl>
Co-Authored-By: Bas Westerbaan <bas@westerbaan.name>

---------

Co-authored-by: Bas Westerbaan <bas@westerbaan.name>
Co-authored-by: Christopher Patton <3453007+cjpatton@users.noreply.github.com>
Co-authored-by: Peter Wu <peter@lekensteyn.nl>

* new: enable PQ parrots (#225)

* Redesign KeySharesEcdheParameters into KeySharesParameters which supports multiple types of keys.

* Optimize program logic to prevent using unwanted keys

* new: more parrots and safety update (#227)

* new: PQ and other parrots

Add new preset parrots:
- HelloChrome_114_Padding_PSK_Shuf
- HelloChrome_115_PQ
- HelloChrome_115_PQ_PSK

* new: ShuffleChromeTLSExtensions

Implement a new function `ShuffleChromeTLSExtensions(exts []TLSExtension) []TLSExtension`.

* update: include psk parameter for parrot-related functions

Update following functions' prototype to accept an optional pskExtension (of type *FakePreSharedKeyExtension):
- `UClient(conn net.Conn, config *Config, clientHelloID ClientHelloID)` => `UClient(conn net.Conn, config *Config, clientHelloID ClientHelloID, pskExtension ...*FakePreSharedKeyExtension)`
- `UTLSIdToSpec(id ClientHelloID)` => `UTLSIdToSpec(id ClientHelloID, pskExtension ...*FakePreSharedKeyExtension)`

* new: pre-defined error from UTLSIdToSpec

Update UTLSIdToSpec to return more comprehensive errors by pre-defining them, allowing easier error comparing/unwrapping.

* new: UtlsPreSharedKeyExtension

In `u_pre_shared_key.go`, create `PreSharedKeyExtension` as an interface, with 3 implementations:
- `UtlsPreSharedKeyExtension` implements full support for `pre_shared_key` less resuming after seeing HRR.
- `FakePreSharedKeyExtension` uses CipherSuiteID, SessionSecret and Identities to calculate the corresponding binders and send them, without setting the internal states. Therefore if the server accepts the PSK and tries to resume, the connection fails.
- `HardcodedPreSharedKeyExtension` allows user to hardcode Identities and Binders to be sent in the extension without setting the internal states. Therefore if the server accepts the PSK and tries to resume, the connection fails.

TODO: Only one of FakePreSharedKeyExtension and HardcodedPreSharedKeyExtension should be kept, the other one should be just removed. We still need to learn more of the safety of hardcoding both Identities and Binders without recalculating the latter.

---------

Co-authored-by: Bas Westerbaan <bas@westerbaan.name>
Co-authored-by: Christopher Patton <3453007+cjpatton@users.noreply.github.com>
Co-authored-by: Peter Wu <peter@lekensteyn.nl>

auth.go

@@ -15,6 +15,9 @@ import (
 	"fmt"
 	"hash"
 	"io"
+
+	circlPki "github.com/cloudflare/circl/pki"
+	circlSign "github.com/cloudflare/circl/sign"
 )
 
 // verifyHandshakeSignature verifies a signature against pre-hashed
@@ -55,7 +58,20 @@ func verifyHandshakeSignature(sigType uint8, pubkey crypto.PublicKey, hashFunc c
 			return err
 		}
 	default:
-		return errors.New("internal error: unknown signature type")
+		// [UTLS SECTION BEGINS]
+		// Ported from cloudflare/go
+		scheme := circlSchemeBySigType(sigType)
+		if scheme == nil {
+			return errors.New("internal error: unknown signature type")
+		}
+		pubKey, ok := pubkey.(circlSign.PublicKey)
+		if !ok {
+			return fmt.Errorf("expected a %s public key, got %T", scheme.Name(), pubkey)
+		}
+		if !scheme.Verify(pubKey, signed, sig, nil) {
+			return fmt.Errorf("%s verification failure", scheme.Name())
+		}
+		// [UTLS SECTION ENDS]
 	}
 	return nil
 }
@@ -106,7 +122,18 @@ func typeAndHashFromSignatureScheme(signatureAlgorithm SignatureScheme) (sigType
 	case Ed25519:
 		sigType = signatureEd25519
 	default:
-		return 0, 0, fmt.Errorf("unsupported signature algorithm: %v", signatureAlgorithm)
+		// [UTLS SECTION BEGINS]
+		// Ported from cloudflare/go
+		scheme := circlPki.SchemeByTLSID(uint(signatureAlgorithm))
+		if scheme == nil {
+			return 0, 0, fmt.Errorf("unsupported signature algorithm: %v", signatureAlgorithm)
+		}
+		sigType = sigTypeByCirclScheme(scheme)
+		if sigType == 0 {
+			return 0, 0, fmt.Errorf("circl scheme %s not supported",
+				scheme.Name())
+		}
+		// [UTLS SECTION ENDS]
 	}
 	switch signatureAlgorithm {
 	case PKCS1WithSHA1, ECDSAWithSHA1:
@@ -120,7 +147,14 @@ func typeAndHashFromSignatureScheme(signatureAlgorithm SignatureScheme) (sigType
 	case Ed25519:
 		hash = directSigning
 	default:
-		return 0, 0, fmt.Errorf("unsupported signature algorithm: %v", signatureAlgorithm)
+		// [UTLS SECTION BEGINS]
+		// Ported from cloudflare/go
+		scheme := circlPki.SchemeByTLSID(uint(signatureAlgorithm))
+		if scheme == nil {
+			return 0, 0, fmt.Errorf("unsupported signature algorithm: %v", signatureAlgorithm)
+		}
+		hash = directSigning
+		// [UTLS SECTION ENDS]
 	}
 	return sigType, hash, nil
 }
@@ -140,6 +174,11 @@ func legacyTypeAndHashFromPublicKey(pub crypto.PublicKey) (sigType uint8, hash c
 		// full signature, and not even OpenSSL bothers with the
 		// complexity, so we can't even test it properly.
 		return 0, 0, fmt.Errorf("tls: Ed25519 public keys are not supported before TLS 1.2")
+	// [UTLS SECTION BEGINS]
+	// Ported from cloudflare/go
+	case circlSign.PublicKey:
+		return 0, 0, fmt.Errorf("tls: circl public keys are not supported before TLS 1.2")
+	// [UTLS SECTION ENDS]
 	default:
 		return 0, 0, fmt.Errorf("tls: unsupported public key: %T", pub)
 	}
@@ -210,6 +249,16 @@ func signatureSchemesForCertificate(version uint16, cert *Certificate) []Signatu
 		}
 	case ed25519.PublicKey:
 		sigAlgs = []SignatureScheme{Ed25519}
+	// [UTLS SECTION BEGINS]
+	// Ported from cloudflare/go
+	case circlSign.PublicKey:
+		scheme := pub.Scheme()
+		tlsScheme, ok := scheme.(circlPki.TLSScheme)
+		if !ok {
+			return nil
+		}
+		sigAlgs = []SignatureScheme{SignatureScheme(tlsScheme.TLSIdentifier())}
+	// [UTLS SECTION ENDS]
 	default:
 		return nil
 	}

auth_test.go

@@ -7,6 +7,8 @@ package tls
 import (
 	"crypto"
 	"testing"
+
+	circlPki "github.com/cloudflare/circl/pki"
 )
 
 func TestSignatureSelection(t *testing.T) {
@@ -161,7 +163,7 @@ func TestSupportedSignatureAlgorithms(t *testing.T) {
 		if sigType == 0 {
 			t.Errorf("%v: missing signature type", sigAlg)
 		}
-		if hash == 0 && sigAlg != Ed25519 {
+		if hash == 0 && sigAlg != Ed25519 && circlPki.SchemeByTLSID(uint(sigAlg)) == nil { // [UTLS] ported from cloudflare/go
 			t.Errorf("%v: missing hash", sigAlg)
 		}
 	}

cfkem.go

@@ -0,0 +1,101 @@
+// Copyright 2022 Cloudflare, Inc. All rights reserved. Use of this source code
+// is governed by a BSD-style license that can be found in the LICENSE file.
+//
+// Glue to add Circl's (post-quantum) hybrid KEMs.
+//
+// To enable set CurvePreferences with the desired scheme as the first element:
+//
+//   import (
+//      "crypto/tls"
+//
+//          [...]
+//
+//   config.CurvePreferences = []tls.CurveID{
+//      tls.X25519Kyber768Draft00,
+//      tls.X25519,
+//      tls.P256,
+//   }
+
+package tls
+
+import (
+	"fmt"
+	"io"
+
+	"crypto/ecdh"
+
+	"github.com/cloudflare/circl/kem"
+	"github.com/cloudflare/circl/kem/hybrid"
+)
+
+// Either *ecdh.PrivateKey or *kemPrivateKey
+type clientKeySharePrivate interface{}
+
+type kemPrivateKey struct {
+	secretKey kem.PrivateKey
+	curveID   CurveID
+}
+
+var (
+	X25519Kyber512Draft00    = CurveID(0xfe30)
+	X25519Kyber768Draft00    = CurveID(0x6399)
+	X25519Kyber768Draft00Old = CurveID(0xfe31)
+	P256Kyber768Draft00      = CurveID(0xfe32)
+	invalidCurveID           = CurveID(0)
+)
+
+// Extract CurveID from clientKeySharePrivate
+func clientKeySharePrivateCurveID(ks clientKeySharePrivate) CurveID {
+	switch v := ks.(type) {
+	case *kemPrivateKey:
+		return v.curveID
+	case *ecdh.PrivateKey:
+		ret, ok := curveIDForCurve(v.Curve())
+		if !ok {
+			panic("cfkem: internal error: unknown curve")
+		}
+		return ret
+	default:
+		panic("cfkem: internal error: unknown clientKeySharePrivate")
+	}
+}
+
+// Returns scheme by CurveID if supported by Circl
+func curveIdToCirclScheme(id CurveID) kem.Scheme {
+	switch id {
+	case X25519Kyber512Draft00:
+		return hybrid.Kyber512X25519()
+	case X25519Kyber768Draft00, X25519Kyber768Draft00Old:
+		return hybrid.Kyber768X25519()
+	case P256Kyber768Draft00:
+		return hybrid.P256Kyber768Draft00()
+	}
+	return nil
+}
+
+// Generate a new shared secret and encapsulates it for the packed
+// public key in ppk using randomness from rnd.
+func encapsulateForKem(scheme kem.Scheme, rnd io.Reader, ppk []byte) (
+	ct, ss []byte, alert alert, err error) {
+	pk, err := scheme.UnmarshalBinaryPublicKey(ppk)
+	if err != nil {
+		return nil, nil, alertIllegalParameter, fmt.Errorf("unpack pk: %w", err)
+	}
+	seed := make([]byte, scheme.EncapsulationSeedSize())
+	if _, err := io.ReadFull(rnd, seed); err != nil {
+		return nil, nil, alertInternalError, fmt.Errorf("random: %w", err)
+	}
+	ct, ss, err = scheme.EncapsulateDeterministically(pk, seed)
+	return ct, ss, alertIllegalParameter, err
+}
+
+// Generate a new keypair using randomness from rnd.
+func generateKemKeyPair(scheme kem.Scheme, curveID CurveID, rnd io.Reader) (
+	kem.PublicKey, *kemPrivateKey, error) {
+	seed := make([]byte, scheme.SeedSize())
+	if _, err := io.ReadFull(rnd, seed); err != nil {
+		return nil, nil, err
+	}
+	pk, sk := scheme.DeriveKeyPair(seed)
+	return pk, &kemPrivateKey{sk, curveID}, nil
+}

cfkem_test.go

@@ -0,0 +1,107 @@
+// Copyright 2022 Cloudflare, Inc. All rights reserved. Use of this source code
+// is governed by a BSD-style license that can be found in the LICENSE file.
+
+package tls
+
+import (
+	"context"
+	"fmt"
+	"testing"
+)
+
+func testHybridKEX(t *testing.T, curveID CurveID, clientPQ, serverPQ,
+	clientTLS12, serverTLS12 bool) {
+	// var clientSelectedKEX *CurveID
+	// var retry bool
+
+	clientConfig := testConfig.Clone()
+	if clientPQ {
+		clientConfig.CurvePreferences = []CurveID{curveID, X25519}
+	}
+	// clientCFEventHandler := func(ev CFEvent) {
+	// 	switch e := ev.(type) {
+	// 	case CFEventTLSNegotiatedNamedKEX:
+	// 		clientSelectedKEX = &e.KEX
+	// 	case CFEventTLS13HRR:
+	// 		retry = true
+	// 	}
+	// }
+	if clientTLS12 {
+		clientConfig.MaxVersion = VersionTLS12
+	}
+
+	serverConfig := testConfig.Clone()
+	if serverPQ {
+		serverConfig.CurvePreferences = []CurveID{curveID, X25519}
+	} else {
+		serverConfig.CurvePreferences = []CurveID{X25519}
+	}
+	if serverTLS12 {
+		serverConfig.MaxVersion = VersionTLS12
+	}
+
+	c, s := localPipe(t)
+	done := make(chan error)
+	defer c.Close()
+
+	go func() {
+		defer s.Close()
+		done <- Server(s, serverConfig).Handshake()
+	}()
+
+	cli := Client(c, clientConfig)
+	// cCtx := context.WithValue(context.Background(), CFEventHandlerContextKey{}, clientCFEventHandler)
+	clientErr := cli.HandshakeContext(context.Background())
+	serverErr := <-done
+	if clientErr != nil {
+		t.Errorf("client error: %s", clientErr)
+	}
+	if serverErr != nil {
+		t.Errorf("server error: %s", serverErr)
+	}
+
+	// var expectedKEX CurveID
+	// var expectedRetry bool
+
+	// if clientPQ && serverPQ && !clientTLS12 && !serverTLS12 {
+	// 	expectedKEX = curveID
+	// } else {
+	// 	expectedKEX = X25519
+	// }
+	// if !clientTLS12 && clientPQ && !serverPQ {
+	// 	expectedRetry = true
+	// }
+
+	// if expectedRetry != retry {
+	// 	t.Errorf("Expected retry=%v, got retry=%v", expectedRetry, retry)
+	// }
+
+	// if clientSelectedKEX == nil {
+	// 	t.Error("No KEX happened?")
+	// } else if *clientSelectedKEX != expectedKEX {
+	// 	t.Errorf("failed to negotiate: expected %d, got %d",
+	// 		expectedKEX, *clientSelectedKEX)
+	// }
+}
+
+func TestHybridKEX(t *testing.T) {
+	run := func(curveID CurveID, clientPQ, serverPQ, clientTLS12, serverTLS12 bool) {
+		t.Run(fmt.Sprintf("%#04x serverPQ:%v clientPQ:%v serverTLS12:%v clientTLS12:%v", uint16(curveID),
+			serverPQ, clientPQ, serverTLS12, clientTLS12), func(t *testing.T) {
+			testHybridKEX(t, curveID, clientPQ, serverPQ, clientTLS12, serverTLS12)
+		})
+	}
+	for _, curveID := range []CurveID{
+		X25519Kyber512Draft00,
+		X25519Kyber768Draft00,
+		X25519Kyber768Draft00Old,
+		P256Kyber768Draft00,
+	} {
+		run(curveID, true, true, false, false)
+		run(curveID, true, false, false, false)
+		run(curveID, false, true, false, false)
+		run(curveID, true, true, true, false)
+		run(curveID, true, true, false, true)
+		run(curveID, true, true, true, true)
+	}
+}

common.go

@@ -189,6 +189,7 @@ const (
 	signatureRSAPSS
 	signatureECDSA
 	signatureEd25519
+	signatureEdDilithium3
 )
 
 // directSigning is a standard Hash value that signals that no pre-hashing
@@ -780,6 +781,11 @@ type Config struct {
 	// its key share in TLS 1.3. This may change in the future.
 	CurvePreferences []CurveID
 
+	// PQSignatureSchemesEnabled controls whether additional post-quantum
+	// signature schemes are supported for peer certificates. For available
+	// signature schemes, see tls_cf.go.
+	PQSignatureSchemesEnabled bool // [UTLS] ported from cloudflare/go
+
 	// DynamicRecordSizingDisabled disables adaptive sizing of TLS records.
 	// When true, the largest possible TLS record size is always used. When
 	// false, the size of TLS records may be adjusted in an attempt to
@@ -885,6 +891,7 @@ func (c *Config) Clone() *Config {
 		MinVersion:                  c.MinVersion,
 		MaxVersion:                  c.MaxVersion,
 		CurvePreferences:            c.CurvePreferences,
+		PQSignatureSchemesEnabled:   c.PQSignatureSchemesEnabled, // [UTLS]
 		DynamicRecordSizingDisabled: c.DynamicRecordSizingDisabled,
 		Renegotiation:               c.Renegotiation,
 		KeyLogWriter:                c.KeyLogWriter,

conn.go

@@ -96,7 +96,7 @@ type Conn struct {
 	// clientProtocol is the negotiated ALPN protocol.
 	clientProtocol string
 
-	utls utlsConnExtraFields // [UTLS】 used for extensive things such as ALPS
+	utls utlsConnExtraFields // [UTLS] used for extensive things such as ALPS, PSK, etc
 
 	// input/output
 	in, out   halfConn