Add option to decrypt Quic traffic in debug builds #83001

wfurt · 2023-03-05T18:41:33Z

essentially

export SSLKEYLOGFILE=/tmp/premaster.txt
projects/h3$ ./bin/Debug/net70/linux-x64/publish/h3 https://www.litespeedtech.com/
cat cat /tmp/premaster.txt

CLIENT_HANDSHAKE_TRAFFIC_SECRET 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 79E0C7E703C685644A8758DEF1E81D37A8408E1759E3D19A31AA09049A2E588A339B17B9EE6BE52ED45ABA3E128D7209
SERVER_HANDSHAKE_TRAFFIC_SECRET 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 7D17DAF6C7FC50ACC15396DC12A8C357DC4B53EF70A0302619523A08811CAD135C1B51EE2AB14E3AFD61ED889A141D5D
CLIENT_TRAFFIC_SECRET_0 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 1E2D5E6E71D4BCEE3DFAA6254AA47843BD0989FFA07B3CBF6E5D9520B598D1A4D1AECC0D230BEF6433F64EBA5CE9A1E6
SERVER_TRAFFIC_SECRET_0 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 61DA6FE1568328F18B34A71BC64D89FAEE556151F820FE5A05038B8F5AE64696A885B215E865B12EBCA439AFEDF858FE

One can feed this file to Wireshark in TLS section

One can inspect Quic handshake as well as the content. Wireshark offers "Follow Quic Stream" to see single H/3 (or other transaction)

This does not impacts shipping product for security reasons.
Tested only on Linux + Wireshark 4.0.4 on MacOS.

ghost · 2023-03-05T18:41:46Z

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

essentially

export SSLKEYLOGFILE=/tmp/premaster.txt
projects/h3$ ./bin/Debug/net70/linux-x64/publish/h3 https://www.litespeedtech.com/
cat cat /tmp/premaster.txt

CLIENT_HANDSHAKE_TRAFFIC_SECRET 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 79E0C7E703C685644A8758DEF1E81D37A8408E1759E3D19A31AA09049A2E588A339B17B9EE6BE52ED45ABA3E128D7209
SERVER_HANDSHAKE_TRAFFIC_SECRET 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 7D17DAF6C7FC50ACC15396DC12A8C357DC4B53EF70A0302619523A08811CAD135C1B51EE2AB14E3AFD61ED889A141D5D
CLIENT_TRAFFIC_SECRET_0 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 1E2D5E6E71D4BCEE3DFAA6254AA47843BD0989FFA07B3CBF6E5D9520B598D1A4D1AECC0D230BEF6433F64EBA5CE9A1E6
SERVER_TRAFFIC_SECRET_0 9831E03D48BB930FC545A4DEF78BC3F2B3F036909066218B0CA2ADB2A2A85CA5 61DA6FE1568328F18B34A71BC64D89FAEE556151F820FE5A05038B8F5AE64696A885B215E865B12EBCA439AFEDF858FE

One can feed this file to Wireshark in TLS section

One can inspect Quic handshake as well as the content. Wireshark offers "Follow Quic Stream" to see single H/3 (or other transaction)

This does not impacts shipping product for security reasons.

Author:	wfurt
Assignees:	wfurt
Labels:	`area-System.Net.Quic`
Milestone:	-

…Secret

wfurt · 2023-03-05T18:45:15Z

BTW if the secret file is updated later then Wireshark deselects the traffic one needs to force that process again. Easiest way IMHO is to ignore/unignore first packet of the stream. Saving and reading from file works as well but the previous method works with life captures.

wfurt · 2023-03-05T19:03:43Z

BTW https://lekensteyn.nl/files/wireshark-ssl-tls-decryption-secrets-sharkfest18eu.pdf is great reading about how it works.

ManickaP · 2023-03-06T09:22:24Z

Apart from the review, I also found this issue: microsoft/msquic#2344. I assume that because of that, this will not fully work on server side, am I right? If so, we should probably put a comment linking the issue if someone stumbles on this.

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

ManickaP · 2023-03-06T10:27:32Z

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

+        }
+    }
+
+    public void WriteSecret(string fileName)


Why the split into 2 public methods?

The screen one will open file for each Quic connection and allocate. And I did not want to drag the stream to QuicConnection right now and I did not want to worry about flushing in log running process. So I split them even if there is currently no benefit. I can merge them together but this was my motivation for keeping them separate.

I'm not sure I understand. WriteSecret is called from 1 place and the parameter-less overload is used. The body of the 2 other overloads could just be collapsed into the parameter-less one, without dragging stream anywhere.

Or did you mean for the stream to be static?

wfurt · 2023-03-07T01:59:08Z

Apart from the review, I also found this issue: microsoft/msquic#2344. I assume that because of that, this will not fully work on server side, am I right?

I was not aware of that. And of course I tested with client. I can perhaps look if I found some spare time.

CarnaViire · 2023-03-10T13:25:56Z

Should we wait until microsoft/msquic#2344 is fixed, or we believe there's value for merging for only the client side now? I'm leaning towards the latter, given that even when msquic's issue will be fixed, it would need to make its way into official release for the feature to be working everywhere... (but we need to add a comment that will mention that for a while it will work only for client)

wfurt · 2023-03-10T19:06:01Z

Server also works on Linux with microsoft/msquic#3481. On Windows, the secret file is missing client random - something one can get from packet capture. Not great but possible.
For development I would think the typical case is that you can control both sides e.g. you can choose where you capture and decrypt.

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

ManickaP · 2023-03-16T15:45:23Z

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

+
+    public unsafe void WriteSecret(FileStream stream)
+    {
+        lock (stream)


Why is here a lock?

I'm not sure if thaw written are really atomic so I'm worried about multiple threads writing entries concurrently for multiple connections.

ManickaP · 2023-03-16T15:45:43Z

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

+        }
+    }
+
+    public void WriteSecret(string fileName)


I'm not sure I understand. WriteSecret is called from 1 place and the parameter-less overload is used. The body of the 2 other overloads could just be collapsed into the parameter-less one, without dragging stream anywhere.

Or did you mean for the stream to be static?

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

…Secret

ghost · 2023-05-04T17:02:12Z

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

…Secret

wfurt · 2023-05-15T19:52:53Z

It took me while to get back to this. There are two big changes:

Once the secret pointer is set, it cannot be unset with current MsQuic. That made the old code problematic when Disposing during handshake e.g. we cannot simply free the memory. To make that easier I added the buffer to the SafeHandle so it has same life cycle as the connection.
With the refactors I lost the original intent to prevent data dangling when multiples secrets are written concurrently from multiple threads. So now there is one FileStream and the lock should work as intended once again.

This should be ready for another review round.

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicSafeHandle.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicSafeHandle.cs

…Secret

wfurt · 2023-06-05T10:38:24Z

I finally got some time to address feedback. This should be ready for another review round.

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs

…Secret

rzikm

LGTM!

Add option to decrypt Quic traffic in debug builds

fdbfbac

wfurt added the area-System.Net.Quic label Mar 5, 2023

wfurt requested review from CarnaViire and ManickaP March 5, 2023 18:41

wfurt self-assigned this Mar 5, 2023

Merge branch 'main' of https://github.com/dotnet/runtime into QuicTls…

6134449

…Secret

build-analysis bot mentioned this pull request Mar 5, 2023

Roslyn source generator crash on mono/linux/arm64 #81123

Closed

ManickaP reviewed Mar 6, 2023

View reviewed changes

wfurt mentioned this pull request Mar 7, 2023

get client random if setting TlsSecrets on server microsoft/msquic#3481

Merged

feedback from review

098ac11

ManickaP reviewed Mar 16, 2023

View reviewed changes

wfurt mentioned this pull request Mar 28, 2023

Support SSLKEYLOGFILE in SslStream #37915

Closed

Merge branch 'main' of https://github.com/dotnet/runtime into QuicTls…

668b905

…Secret

wfurt marked this pull request as draft April 4, 2023 15:14

ghost closed this May 4, 2023

fixes & feedback

da206a6

wfurt reopened this May 15, 2023

Merge branch 'main' of https://github.com/dotnet/runtime into QuicTls…

61fb5e4

…Secret

wfurt marked this pull request as ready for review May 15, 2023 19:45

ManickaP reviewed May 16, 2023

View reviewed changes

CarnaViire reviewed May 22, 2023

View reviewed changes

src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs Outdated Show resolved Hide resolved

CarnaViire reviewed May 22, 2023

View reviewed changes

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs Show resolved Hide resolved

CarnaViire reviewed May 22, 2023

View reviewed changes

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicSafeHandle.cs Show resolved Hide resolved

karelz added this to the 8.0.0 milestone Jun 1, 2023

wfurt added 2 commits June 2, 2023 02:22

Merge branch 'main' of https://github.com/dotnet/runtime into QuicTls…

977b275

…Secret

feedback from review

2cfe28f

build-analysis bot mentioned this pull request Jun 5, 2023

Tracking issue for CI build timeouts #76454

Closed

rzikm reviewed Jun 6, 2023

View reviewed changes

src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/MsQuicTlsSecret.cs Show resolved Hide resolved

Merge branch 'main' of https://github.com/dotnet/runtime into QuicTls…

e3f25df

…Secret

rzikm approved these changes Jul 3, 2023

View reviewed changes

ManickaP approved these changes Jul 3, 2023

View reviewed changes

wfurt merged commit 3be1483 into dotnet:main Jul 5, 2023
103 checks passed

wfurt deleted the QuicTlsSecret branch July 5, 2023 21:17

ghost locked as resolved and limited conversation to collaborators Aug 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add option to decrypt Quic traffic in debug builds #83001

Add option to decrypt Quic traffic in debug builds #83001

wfurt commented Mar 5, 2023 •

edited

Loading

ghost commented Mar 5, 2023

wfurt commented Mar 5, 2023

wfurt commented Mar 5, 2023

ManickaP commented Mar 6, 2023

ManickaP Mar 6, 2023

wfurt Mar 7, 2023

ManickaP Mar 16, 2023

wfurt commented Mar 7, 2023

CarnaViire commented Mar 10, 2023

wfurt commented Mar 10, 2023

ManickaP Mar 16, 2023

wfurt Jun 5, 2023

ManickaP Mar 16, 2023

ghost commented May 4, 2023

wfurt commented May 15, 2023

wfurt commented Jun 5, 2023

rzikm left a comment

Add option to decrypt Quic traffic in debug builds #83001

Add option to decrypt Quic traffic in debug builds #83001

Conversation

wfurt commented Mar 5, 2023 • edited Loading

ghost commented Mar 5, 2023

wfurt commented Mar 5, 2023

wfurt commented Mar 5, 2023

ManickaP commented Mar 6, 2023

ManickaP Mar 6, 2023

Choose a reason for hiding this comment

wfurt Mar 7, 2023

Choose a reason for hiding this comment

ManickaP Mar 16, 2023

Choose a reason for hiding this comment

wfurt commented Mar 7, 2023

CarnaViire commented Mar 10, 2023

wfurt commented Mar 10, 2023

ManickaP Mar 16, 2023

Choose a reason for hiding this comment

wfurt Jun 5, 2023

Choose a reason for hiding this comment

ManickaP Mar 16, 2023

Choose a reason for hiding this comment

ghost commented May 4, 2023

wfurt commented May 15, 2023

wfurt commented Jun 5, 2023

rzikm left a comment

Choose a reason for hiding this comment

wfurt commented Mar 5, 2023 •

edited

Loading