Use custom validation callback for server certificates in HTTP handlers #6665

simonrozsival · 2022-01-25T16:07:30Z

The AndroidClientHandler and AndroidMessageHandler classes both have the ServerCertificateCustomValidationCallback property, which should be useful e.g. to allow running the Android app against a server with a self-signed SSL certificate during development, but the callback is never used. Unfortunatelly since .NET 6 the System.Net.Http.SocketsHttpHandler for Android doesn't support the use case anymore. That means that the recommended way of connecting to local web server won't work in MAUI.

This PR introduces an implementation of IX509TrustManger which wraps the default Java X509 trust manager and calls the user's callback on top of the default validation.

The Legacy APK test works just fine.
The .NET APK test doesn't even build and I'm not sure what to do about it. I'd be grateful for any pointers to setup of .NET 6 APK tests. - I removed the .NET test as I couldn't get it to work.
It turns out that X509Chain Build function doesn't work on Android, so I'm not calling it and I'm passing the chain to the callback directly. ~~I'm leaving the TODO in the source for now and maybe somebody will have an idea what to do about it during reviews.~~

Ref dotnet/runtime#62966

…simonrozsival/fix-android-message-handler-server-certificate-custom-validation-callback

src/Mono.Android/Xamarin.Android.Net/X509TrustManagerWithValidationCallback.cs

…simonrozsival/android-http-handlers-use-server-certificate-custom-validation-callback

This reverts commit 90f4bf2.

This reverts commit 491d1c1.

simonrozsival · 2022-04-28T10:23:15Z

@jonathanpeppers @jonpryo I've moved the code into tests/Mono.Android-Tests but now the tests are failing because the new class can't be instantiated:

Java.Lang.ClassNotFoundException : xamarin.android.net.X509TrustManagerWithValidationCallback
----> Java.Lang.ClassNotFoundException : xamarin.android.net.X509TrustManagerWithValidationCallback
 at Java.Interop.JniEnvironment.StaticMethods.CallStaticObjectMethod(JniObjectReference , JniMethodInfo , JniArgumentValue* )
   at Android.Runtime.JNIEnv.FindClass(String )
   at Android.Runtime.JNIEnv.AllocObject(String )
   at Android.Runtime.JNIEnv.StartCreateInstance(String , String , JValue* )
   at Android.Runtime.JNIEnv.StartCreateInstance(String , String , JValue[] )
   at Xamarin.Android.Net.X509TrustManagerWithValidationCallback..ctor(IX509TrustManager , HttpRequestMessage , Func`5 )
   at Xamarin.Android.Net.X509TrustManagerWithValidationCallback.Helper.Inject(ITrustManager[] , HttpRequestMessage )
   at Xamarin.Android.Net.AndroidMessageHandler.SetupSSL(HttpsURLConnection , HttpRequestMessage )
   at Xamarin.Android.Net.AndroidMessageHandler.SetupRequestInternal(HttpRequestMessage , URLConnection )
   at Xamarin.Android.Net.AndroidMessageHandler.SendAsync(HttpRequestMessage , CancellationToken )
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage , CancellationToken )
   at NUnit.Framework.Internal.AsyncInvocationRegion.AsyncTaskInvocationRegion.WaitForPendingOperationsToComplete(Object invocationResult)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.RunAsyncTestMethod(TestExecutionContext context)
  --- End of managed Java.Lang.ClassNotFoundException stack trace ---
java.lang.ClassNotFoundException: xamarin.android.net.X509TrustManagerWithValidationCallback
	at java.lang.Class.classForName(Native Method)
	at java.lang.Class.forName(Class.java:454)
	at crc643df67da7b13bb6b1.TestInstrumentation_1.n_onStart(Native Method)
	at crc643df67da7b13bb6b1.TestInstrumentation_1.onStart(TestInstrumentation_1.java:35)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)
Caused by: java.lang.ClassNotFoundException: xamarin.android.net.X509TrustManagerWithValidationCallback
	... 5 more

--ClassNotFoundException

  --- End of managed Java.Lang.ClassNotFoundException stack trace ---
java.lang.ClassNotFoundException: xamarin.android.net.X509TrustManagerWithValidationCallback
	at java.lang.Class.classForName(Native Method)
	at java.lang.Class.forName(Class.java:454)
	at crc643df67da7b13bb6b1.TestInstrumentation_1.n_onStart(Native Method)
	at crc643df67da7b13bb6b1.TestInstrumentation_1.onStart(TestInstrumentation_1.java:35)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)

I don't know how make it work. On Discord you said that the classes in Mono.Android are "special cased". I wasn't able to find a way to special case the X509TrustManagerWithValidationCallback on my own. Could you please help me finish this PR or point me in the right direction? I believe this is the last obstacle for this PR since @akoeplinger figured out how to get rid fo the excess file size when this feature isn't used.

simonrozsival · 2022-04-28T15:47:14Z

I resolved the problem I was having. It was caused by missing Proguard configuration.

…ozsival/android-http-handlers-use-server-certificate-custom-validation-callback

jonathanpeppers · 2022-04-29T15:08:55Z

src/Xamarin.Android.Build.Tasks/Resources/proguard_xamarin.cfg

@@ -15,6 +15,7 @@
 -keep class opentk_1_0.platform.android.AndroidGameView { *; <init>(...); }
 -keep class opentk_1_0.GameViewBase { *; <init>(...); }
 -keep class com.xamarin.java_interop.ManagedPeer { *; <init>(...); }
+-keep class xamarin.android.net.X509TrustManagerWithValidationCallback { *; <init>(...); }


So it looks like this is needed because of:

https://github.com/xamarin/xamarin-android/blob/46002b49d8c0b7b1a17532a8e104b4d31afee7a6/src/Xamarin.Android.Build.Tasks/Linker/MonoDroid.Tuner/GenerateProguardConfiguration.cs#L50-L57

Mono.Android.dll doesn't reference itself, so it's skipped. We should probably go with your new rule here, and I might address this in a future PR.

…HTTP handlers (dotnet#6665) Backport of: dotnet#6665 Context: dotnet/runtime#62966 The `AndroidClientHandler` and `AndroidMessageHandler` classes both have the `ServerCertificateCustomValidationCallback` property, which should be useful e.g. to allow running the Android app against a server with a self-signed SSL certificate during development, but the callback is never used. Unfortunatelly since .NET 6 the `System.Net.Http.SocketsHttpHandler` for Android doesn't support the use case anymore. That means that [the recommended way of connecting to local web server][0] won't work in MAUI. This PR introduces an implementation of `IX509TrustManger` which wraps the default Java X509 trust manager and calls the user's callback on top of the default validation. It turns out that `X509Chain` `Build` function doesn't work on Android, so I'm not calling it and I'm passing the chain to the callback directly. Additionally, we need a default proguard rule due to: https://github.com/xamarin/xamarin-android/blob/46002b49d8c0b7b1a17532a8e104b4d31afee7a6/src/Xamarin.Android.Build.Tasks/Linker/MonoDroid.Tuner/GenerateProguardConfiguration.cs#L50-L57 -keep class xamarin.android.net.X509TrustManagerWithValidationCallback { *; <init>(...); } `Mono.Android.dll` is skipped during the `GenerateProguardConfiguration` linker step. It might be worth addressing this in a future PR. [0]: https://docs.microsoft.com/en-us/xamarin/cross-platform/deploy-test/connect-to-local-web-services

…HTTP handlers (#6972) * [Mono.Android] custom validation callback for server certificates in HTTP handlers (#6665) Backport of: #6665 Context: dotnet/runtime#62966 The `AndroidClientHandler` and `AndroidMessageHandler` classes both have the `ServerCertificateCustomValidationCallback` property, which should be useful e.g. to allow running the Android app against a server with a self-signed SSL certificate during development, but the callback is never used. Unfortunatelly since .NET 6 the `System.Net.Http.SocketsHttpHandler` for Android doesn't support the use case anymore. That means that [the recommended way of connecting to local web server][0] won't work in MAUI. This PR introduces an implementation of `IX509TrustManger` which wraps the default Java X509 trust manager and calls the user's callback on top of the default validation. It turns out that `X509Chain` `Build` function doesn't work on Android, so I'm not calling it and I'm passing the chain to the callback directly. Additionally, we need a default proguard rule due to: https://github.com/xamarin/xamarin-android/blob/46002b49d8c0b7b1a17532a8e104b4d31afee7a6/src/Xamarin.Android.Build.Tasks/Linker/MonoDroid.Tuner/GenerateProguardConfiguration.cs#L50-L57 -keep class xamarin.android.net.X509TrustManagerWithValidationCallback { *; <init>(...); } `Mono.Android.dll` is skipped during the `GenerateProguardConfiguration` linker step. It might be worth addressing this in a future PR. [0]: https://docs.microsoft.com/en-us/xamarin/cross-platform/deploy-test/connect-to-local-web-services * Update .apkdesc (probably merge issue) Co-authored-by: Simon Rozsival <simon@rozsival.com>

lmageste · 2022-06-01T22:52:43Z

Hi @simonrozsival and @jonathanpeppers

I am consuming this change in the latest version of Visual Studio Preview 17.3. Preview 1.1 using Maui (.NET 6).

I can confirm that the test using the badssl website with self-signed certificate works and gets correctly bypassed, thanks for the change!

In fact, every common certificate problem they have in their website works with the validation callback bypassing cert errors, except for the wrong host example

This scenario, along with all others, were working before, but this particular one is, unfortunately, a blocker for me. I believe you will be able to repro it on your side as well by attempting to access that URL.

Follows the exception I get after using the validation callback to bypass the certificate errors in the wrong host website from badssl:
System.Net.WebException
Message=Hostname wrong.host.badssl.com not verified:
certificate: sha1/1fBSv4IoEephjuel/BGL9TBSEgk=
DN: CN=.badssl.com
subjectAltNames: [.badssl.com, badssl.com]

(Please let me know if I should file another bug for this particular case)

simonrozsival · 2022-06-02T07:08:25Z

Hi @lmageste! I think you need to extend AndroidMessageHandler with your own class and overwrite GetSSLHostnameVerifier and return your own implementation of IHostnameVerifier which will bypass that validation. Please try it and let me know if that helps or not (see https://stackoverflow.com/a/15252178).

lmageste · 2022-06-08T17:42:45Z

Thanks @simonrozsival!

This works and I can communicate with the server with a wrong host certificate. However, it is not ideal since I need to use a platform-specific API (Android) in order to extend HostnameVerifier and establish communication.

Ideally, since my app targets multi-platforms, a universal solution for my http client library targeting a single framework would be the best (my http client communication library is universal and consumed by all of my platforms), which was how it worked before in the Android platform (when using mono android 8.1 framework) by simply extending the ServerCertificateCustomValidationCallback.

This is, although a bit cumbersome (since my client library needs to target android specifically for this workaround), an acceptable workaround for my case, however.

Thank again!

simonrozsival · 2022-06-09T09:17:26Z

The certificate validation and hostname verification are two independent checks. I'm afraid that in your case you really need both and you'll have to have some platform-specific code in your app. I'm not sure why it wasn't necessary before. Is it possible that you used the managed implementation of the HTTP handler instead of the native one?

dotnet/runtime#62966 dotnet/android#6665

Eilon · 2022-06-23T18:48:33Z

I started a discussion topic on how to connect from Android emulators to a local ASP.NET Web API running on Windows: dotnet/maui#8131

Please check that out and let us know if you have any feedback on any of the solutions presented.

…ack usage in NSUrlSessionHandler (#15117) We recently implemented the `ServerCertificateCustomValidationCallback` in Xamarin.Android (dotnet/android#6665). It would be great to have feature parity and support the same callback in Xamarin.iOS and Xamarin.Mac. Related to dotnet/runtime#68898. Partial fix for #14632. Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com> Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

simonrozsival added 13 commits January 24, 2022 17:39

Add basic implementaiton of custom X509 Trust Manager

42c0dc5

Merge branch 'main' of git://github.com/xamarin/xamarin-android into …

cf11487

…simonrozsival/fix-android-message-handler-server-certificate-custom-validation-callback

Update the implementaiton of the legacy http client handler

ad3faad

Update chain building

3637e2f

Add integration tests

fb47fc2

Fix typo

478cc55

Fix Func signature

1579766

Fix Func signature in tests

7976be8

Fix incorrect return type of async tests

6b466f2

Add missing usings to tests

e7f8303

Fix tests

2e96d0a

Fix func generic params nullability

d19208b

Attempt to fix trust manager instantiation

2d102dc

jonpryor reviewed Jan 27, 2022

View reviewed changes

src/Mono.Android/Xamarin.Android.Net/X509TrustManagerWithValidationCallback.cs Outdated Show resolved Hide resolved

simonrozsival added 16 commits January 27, 2022 11:37

Change spaces to tabs

f0dedda

Remove Register attribute

73ac4b8

Remove integration tests

32b5ae3

WIP separate apk for net tests

728d0d8

Finalize tests

077d51a

Update implementation for both handlers

98f13da

Merge branch 'main' of git://github.com/xamarin/xamarin-android into …

d6f4099

…simonrozsival/android-http-handlers-use-server-certificate-custom-validation-callback

Adjust newlines

bcf498a

Fix project and solution

82371c2

Fix Func return value

6d96d0a

Add missing permission

25f81a4

Update test project settings

f0265ed

Update comments

c2a7454

Remove TODOs

d3acfd1

Try to add .net6 tests

31daa4b

Rename csproj files

f23eaa0

simonrozsival added 3 commits April 27, 2022 11:20

WIP try to preserve X509TrustManagerWithValidationCallback

90f4bf2

Revert "WIP try to preserve X509TrustManagerWithValidationCallback"

ac0a75c

This reverts commit 90f4bf2.

Revert "Add the Register attribute"

cd03384

This reverts commit 491d1c1.

simonrozsival added 3 commits April 28, 2022 14:37

Fix the type of exception that we catch

19fca32

Add proguard rule

0d61693

Fix several tests

25f7961

Merge branch 'main' of github.com:xamarin/xamarin-android into simonr…

a91b69c

…ozsival/android-http-handlers-use-server-certificate-custom-validation-callback

simonrozsival requested a review from jonathanpeppers April 28, 2022 21:28

jonathanpeppers approved these changes Apr 29, 2022

View reviewed changes

jonathanpeppers merged commit 48540d6 into dotnet:main Apr 29, 2022

jonathanpeppers mentioned this pull request Apr 29, 2022

[Mono.Android] custom validation callback for server certificates in HTTP handlers #6972

Merged

simonrozsival mentioned this pull request May 5, 2022

[Android][iOS] Enable HttpClientHandler.DangerousAcceptAnyServerCertificateValidator when using native handlers dotnet/runtime#68898

Closed

simonrozsival mentioned this pull request May 25, 2022

[Foundation] Implement the server certificate custom validation callback usage in NSUrlSessionHandler xamarin/xamarin-macios#15117

Merged

simonrozsival deleted the simonrozsival/android-http-handlers-use-server-certificate-custom-validation-callback branch June 9, 2022 09:13

davidbritch mentioned this pull request Jun 15, 2022

Document how to work with a local ASP.NET Core backend during development dotnet/docs-maui#600

Closed

danch15 added a commit to danch15/MauiAndDevDemo that referenced this pull request Jun 16, 2022

https://github.com/dotnet/runtime/issues/62966

293a410

dotnet/runtime#62966 dotnet/android#6665

simonrozsival mentioned this pull request Aug 10, 2022

[Mono.Android] Improve server certificate custom validation callback integration to cover hostname mismatch #7246

Merged

omghb mentioned this pull request Aug 23, 2022

X509Chain.Build is not working #7299

Closed

omghb mentioned this pull request Dec 15, 2022

Custom validation callback for server certificates in SslStream does not work #7641

Closed

github-actions bot locked and limited conversation to collaborators Jan 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use custom validation callback for server certificates in HTTP handlers #6665

Use custom validation callback for server certificates in HTTP handlers #6665

simonrozsival commented Jan 25, 2022 •

edited

Loading

simonrozsival commented Apr 28, 2022

simonrozsival commented Apr 28, 2022

jonathanpeppers Apr 29, 2022

lmageste commented Jun 1, 2022

simonrozsival commented Jun 2, 2022 •

edited

Loading

lmageste commented Jun 8, 2022

simonrozsival commented Jun 9, 2022

Eilon commented Jun 23, 2022

Use custom validation callback for server certificates in HTTP handlers #6665

Use custom validation callback for server certificates in HTTP handlers #6665

Conversation

simonrozsival commented Jan 25, 2022 • edited Loading

simonrozsival commented Apr 28, 2022

simonrozsival commented Apr 28, 2022

jonathanpeppers Apr 29, 2022

Choose a reason for hiding this comment

lmageste commented Jun 1, 2022

simonrozsival commented Jun 2, 2022 • edited Loading

lmageste commented Jun 8, 2022

simonrozsival commented Jun 9, 2022

Eilon commented Jun 23, 2022

simonrozsival commented Jan 25, 2022 •

edited

Loading

simonrozsival commented Jun 2, 2022 •

edited

Loading