refactor(core): alpha.7 rewrite #574

sinui0 · 2024-08-27T05:46:07Z

This PR completely rewrites tlsn-core for our alpha.7 release. Closes #438

There are still a couple of loose ends that I'm tidying up but the vast majority of the deliverable is there. Prior to accepting this PR we will need to review our new fork of rs_merkle which was required to support hash algorithm customization.

A follow up PR will be provided which completes the integration of these changes into the other crates. I have most of this integration work complete on a separate branch already.

Highlights

Improved the structure of attestations
- Header has a static smol size and now includes a version tag and unique identifier
- Body fields are now merkelized so that only fields necessary to prove desired statements need to be included
- Body is forward-compatible with new kinds of fields (for example, new kinds of commitments)
Added a Request pattern which will allow a Prover to configure aspects of the attestation depending on what the Notary supports.
- This paves the way for custom extensions, eg Support custom extensions in attestation #342
Further encapsulated various kinds of proofs
- ServerIdentityProof proves the server name using the certificate chain
- AttestationProof proves the Notary signature and attestation body
- TranscriptProof proves subsequences of the transcript using the encoding or hash commitments.
Extensible cryptographic algorithms via CryptoProvider (inspired by rustls).
- Supports multiple hash algorithms, including user provided custom.
- Supports multiple signature algorithms, including user provided custom.
Added a new PlaintextHash commitment type (not fully exposed yet) which will be implemented with authdecode. See Authdecode #479
Encapsulated more logic into this crate in the form of builders, reducing API surface area and making it more difficult to misuse. There should now be very little implementation code in prover and verifier crates related to attestations.

Not Included

There are some items that I initially wanted in scope but decided to remove as they proved to be larger than anticipated.

Protobufs
Canonical serialization format which is efficient to implement in the EVM. For now we continue to use BCS which uses varint encoding. This may actually be ok, but I've not looked into it.
Extensive versioning
Support custom extensions in attestation #342
Notary identity features #351

th4s

🚀 Great work 🚀

Overall looking very good! Most of the complexity is encapsulated and organized well into modules. Apart from the remaining todos and clippy comments I think the only thing needed is more unit tests.

crates/data-fixtures/data/http/response_json

crates/core/src/attestation.rs

crates/core/src/hash.rs

crates/core/src/index.rs

crates/core/src/signing.rs

crates/core/src/transcript/commit.rs

crates/core/src/transcript/encoding/provider.rs

crates/core/tests/api.rs

themighty1 · 2024-09-09T13:12:40Z

I have e general question while still re-viewing this PR. Some time ago we started using the "TLS peer" terminology since we allow for the TLS Server to act as a TLSN Prover.
This PR goes back to using the "Prover==TLS Client" terminology, e.g. by calling the type "ServerCertCommitment" instead of "TLSPeerCertCommitment".
Is that intended?

maceip · 2024-09-11T22:51:13Z

ServerIdentityProof proves the server name using the certificate chain

AttestationProof proves the Notary signature and attestation body

TranscriptProof proves subsequences of the transcript using the encoding or hash commitments.

👨‍🍳

crates/core/src/presentation.rs

crates/core/src/signing.rs

crates/core/src/connection.rs

crates/core/src/transcript.rs

sinui0 · 2024-09-16T16:01:57Z

I have e general question while still re-viewing this PR. Some time ago we started using the "TLS peer" terminology since we allow for the TLS Server to act as a TLSN Prover. This PR goes back to using the "Prover==TLS Client" terminology, e.g. by calling the type "ServerCertCommitment" instead of "TLSPeerCertCommitment". Is that intended?

For now I think those semantics will only contribute to confusion. In the future can always add a field for client certificate commitment.

* refactor(core): alpha.7 rewrite * allow empty idx * fix empty assumption * further encapsulate rangeset * added presentation, finishing touches * remove unwrap * refactor(core): integrate rewrite changes * remove obsolete tests * add secp256r1 support * update index naming * add secp256r1 support * add attestation to presentation output, and serde derives * handle k256 in KeyAlgId Display * unnecessary newline * fix variable name

sinui0 · 2024-09-18T20:17:24Z

Let's get this merged! Only examples are failing AFAICT, will fix in follow up PRs

th4s

🚀 nice

themighty1

.

themighty1 · 2024-10-21T11:49:01Z

crates/prover/src/prove.rs

+        self.state
+            .mux_fut
+            .poll_with(async {
+                // Send the partial transcript to the verifier.


@sinui0 , what is the motivation here for sending a zeroed out transcript to the verifier? We could introduce a new type which doesn't store the 0s.

This crossed my mind, I just moved it out of scope of the PR. A good solution is to write a custom serializer/deserializer which removes the 0s instead of adding a new type

themighty1 · 2024-10-21T13:41:05Z

ACK, including our rs-merkle fork, (will open a PR with some fixes)
@sinui0 could you pls answer my q above.

sinui0 requested review from themighty1 and th4s August 27, 2024 05:46

th4s reviewed Sep 3, 2024

View reviewed changes

yuroitaki self-requested a review September 9, 2024 03:13

sinui0 mentioned this pull request Sep 11, 2024

refactor(core): integrate rewrite changes #584

Merged

maceip reviewed Sep 12, 2024

View reviewed changes

crates/core/src/presentation.rs Show resolved Hide resolved

yuroitaki reviewed Sep 12, 2024

View reviewed changes

crates/core/src/signing.rs Show resolved Hide resolved

crates/core/src/signing.rs Show resolved Hide resolved

yuroitaki reviewed Sep 13, 2024

View reviewed changes

crates/core/src/connection.rs Show resolved Hide resolved

crates/core/src/transcript.rs Outdated Show resolved Hide resolved

sinui0 added 11 commits September 17, 2024 21:36

refactor(core): alpha.7 rewrite

7947749

allow empty idx

29092cf

fix empty assumption

404cd04

further encapsulate rangeset

75f42ba

added presentation, finishing touches

d695924

remove unwrap

24ec72b

update index naming

1ec34cc

add secp256r1 support

f642986

add attestation to presentation output, and serde derives

a90d6e0

handle k256 in KeyAlgId Display

a69dc97

unnecessary newline

5118309

sinui0 force-pushed the refactor/core-rewrite branch from bc75ed9 to 5118309 Compare September 18, 2024 04:36

sinui0 mentioned this pull request Sep 18, 2024

Document tlsn-core #590

Closed

sinui0 added 2 commits September 18, 2024 09:42

restore changes from dev to tlsn-prover

2a8f42d

heeckhau added this to the Alpha7 milestone Sep 18, 2024

sinui0 added 3 commits September 18, 2024 09:59

use CryptoProvider in config

aa5d528

clippy

6646083

more clippy

5655629

sinui0 requested review from th4s and yuroitaki September 18, 2024 20:16

th4s approved these changes Sep 19, 2024

View reviewed changes

sinui0 merged commit 53ff873 into dev Sep 19, 2024
3 of 5 checks passed

sinui0 deleted the refactor/core-rewrite branch September 19, 2024 14:57

sinui0 mentioned this pull request Sep 19, 2024

Support arbitrary Notary signature schemes #438

Closed

sinui0 mentioned this pull request Oct 10, 2024

Notary identity features #351

Closed

themighty1 reviewed Oct 21, 2024

View reviewed changes

themighty1 restored the refactor/core-rewrite branch October 24, 2024 12:44

themighty1 deleted the refactor/core-rewrite branch October 24, 2024 13:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(core): alpha.7 rewrite #574

refactor(core): alpha.7 rewrite #574

sinui0 commented Aug 27, 2024 •

edited

Loading

th4s left a comment

themighty1 commented Sep 9, 2024

maceip commented Sep 11, 2024

sinui0 commented Sep 16, 2024 •

edited

Loading

sinui0 commented Sep 18, 2024

th4s left a comment

themighty1 left a comment

themighty1 Oct 21, 2024

sinui0 Oct 21, 2024

themighty1 commented Oct 21, 2024 •

edited

Loading

refactor(core): alpha.7 rewrite #574

refactor(core): alpha.7 rewrite #574

Conversation

sinui0 commented Aug 27, 2024 • edited Loading

Highlights

Not Included

th4s left a comment

Choose a reason for hiding this comment

themighty1 commented Sep 9, 2024

maceip commented Sep 11, 2024

sinui0 commented Sep 16, 2024 • edited Loading

sinui0 commented Sep 18, 2024

th4s left a comment

Choose a reason for hiding this comment

themighty1 left a comment

Choose a reason for hiding this comment

themighty1 Oct 21, 2024

Choose a reason for hiding this comment

sinui0 Oct 21, 2024

Choose a reason for hiding this comment

themighty1 commented Oct 21, 2024 • edited Loading

sinui0 commented Aug 27, 2024 •

edited

Loading

sinui0 commented Sep 16, 2024 •

edited

Loading

themighty1 commented Oct 21, 2024 •

edited

Loading