From 18fd1d5393898aa40797b14b0a37c491d0947013 Mon Sep 17 00:00:00 2001 From: Marek Habersack Date: Tue, 14 Jan 2020 22:05:17 +0100 Subject: [PATCH] New code to perform managed <-> java lookups (typemap) Xamarin.Assembly needs to "translate" managed types to Java types and vice versa in order to provide a bridge between the two world. So far it has been done using a straightforward (and fast) method of performing the lookups - all the type pairs were stored in two tables of the same size, with all type names padded to the width of the longest name so that the `bsearch` C function can be used to quickly perform a binary search over the data set. This approach works very well at the expense of data size (shorter strings are 0-padded to the maximum width) and a slightly degraded performace because of the requirement to perform string comparisons. Furthermore, the lookup required that reflection is used to obtain full managed type name (when translating from managed to Java) or to get a `Type` instance from type name (when translating from Java to managed). For Release builds all the above data is placed in the `libxamarin-app.so` library, for Debug builds it is also placed in two files - one for each direction of lookup, described above. This commit is a slight improvement over the above scheme. It eliminates reflection from the process by using managed type tokens (which are integers) and using UUID/Guid of the module in which the type is found. This allows us to perform the binary search over the set of 20 bytes (16 bytes for the UUID and 4 bytes for the token ID) for managed to Java lookups and a single string comparison + binary search over a set of integers for the Java to managed lookup. Java type names must still be used because Java doesn't provide any equivalent to the .NET's type token and module UUID. Those names are still 0-padded to the width of the longest name but there are no longer duplicated. Managed type names are eliminated completely. If Xamarin.Android Instant Run is not used (which is the case for OSS code) for Debug builds, the operation is performed in the same way for both Release and Debug builds. If, however, Instant Run is in effect, the type maps are stored in several files with the .typemap extension - one per **module**. The files contain both the Java to managed maps as well as managed to Java maps (which use indexes into the Java to managed maps). All of those files are loaded during Debug app startup and used to construct a dataset which is the searched during all the lookups. Typemap index file format, all data is little-endian: ---- **Header format** `[Magic string]` # XATI `[Format version]` # 32-bit unsigned integer, 4 bytes `[Entry count]` # 32-bit unsigned integer, 4 bytes `[Module file name width]` # 32-bit unsigned integer, 4 bytes `[Index entries]` # Format described below, `Entry count` entries **Index entry format:** `[Module UUID][File name]` *Where:* `[Module UUID]` is 16 bytes long `[File name]` is right-padded with `` characters to the `[Module file name width]` boundary. Typemap file format, all data is little-endian: ---- **Header format** `[Magic string]` # XATM `[Format version]` # 32-bit integer, 4 bytes `[Module UUID]` # 16 bytes `[Entry count]` # unsigned 32-bit integer, 4 bytes `[Duplicate count]` # unsigned 32-bit integer, 4 bytes (might be 0) `[Java type name width]` # unsigned 32-bit integer, 4 bytes `[Assembly name size]` # unsigned 32-bit integer, 4 bytes `[Assembly name]` # Non-null terminated assembly name `[Java-to-managed map]` # Format described below, `[Entry count]` entries `[Managed-to-java map]` # Format described below, `[Entry count]` entries `[Managed-to-java duplicates map]` # Map of unique managed IDs which point to the same Java type name (might be empty) **Java-to-managed map format:** `[Java type name][Managed type token ID]` Each name is padded with `` to the width specified in the `[Java type name width]` field above. Names are written without the size prefix, instead they are always terminated with a nul character to make it easier and faster to handle by the native runtime. Each token ID is an unsigned 32-bit integer, 4 bytes **Managed-to-java map format:** `[Managed type token ID][Java type name table index]` Both fields are unsigned 32-bit integers, to a total of 8 bytes per entry. Index points into the `[Java-to-managed map]` table above. **Managed-to-java duplicates map format:** Format is identical to `[Managed-to-java]` above. Size changes (XF integration test, `libxamarin-app.so`, Release build): ---- - armeabi-v7a - before: 376616 - after: 97860 - arm64-v8a - before: 377408 - after: 104192 - x86 - before: 376424 - after: 97604 Performance changes (XF integration test, Release build): ---- Device name: **Pixel 3 XL** Device architecture: **arm64-v8a** Number of test runs: **10** | | **Native to managed** | **Runtime init** | **Displayed** | **Notes** | |-----------------|------------------------|------------------|---------------|--------------------------------| | **master** | 141.102 | 160.606 | 839.80 | preload enabled; 32-bit build | | **this commit** | 134.539 | 154.701 | 836.10 | | | **master** | 141.743 | 158.325 | 837.20 | preload disabled; 32-bit build | | **this commit** | 134.064 | 149.137 | 831.90 | | | **master** | 134.526 | 152.640 | 805.10 | preload enabled; 64-bit build | | **this commit** | 126.376 | 143.226 | 788.60 | | | **master** | 134.049 | 149.543 | 779.40 | preload disabled; 64-bit build | | **this commit** | 124.847 | 139.227 | 776.10 | | Build performance (**Release** build): ---- **Before** 389 ms GenerateJavaStubs 1 calls **After** 247 ms GenerateJavaStubs 1 calls New code generates only native assembly or only the binary typemap files, unlike the old code which generated both. Initially the new generator code was moved to a separate task, but Jonathan Peppers determined that it was suboptimal and re-integrated the code back with `GenerateJavaStubs` --- Documentation/guides/messages/README.md | 1 + Documentation/guides/messages/xa4308.md | 18 + .../Steps/Step_DownloadMonoArchive.cs | 50 +- .../Android.Runtime/AndroidRuntime.cs | 12 +- src/Mono.Android/Android.Runtime/JNIEnv.cs | 53 +- src/Mono.Android/Java.Interop/TypeManager.cs | 16 +- .../Properties/AssemblyInfo.cs.in | 4 + .../Java.Interop-Tests.csproj | 5 +- .../Properties/AssemblyInfo.cs | 30 - .../Test/Java.Interop/JnienvTest.cs | 26 +- .../Mono.Android-Test.Library.csproj | 1 - .../Properties/AssemblyInfo.cs | 30 - .../Test/Mono.Android-Tests.csproj | 2 + .../Android/Xamarin.Android.Designer.targets | 12 +- .../Properties/Resources.Designer.cs | 8 + .../Properties/Resources.resx | 5 +- .../Properties/xlf/Resources.cs.xlf | 5 + .../Properties/xlf/Resources.de.xlf | 5 + .../Properties/xlf/Resources.es.xlf | 5 + .../Properties/xlf/Resources.fr.xlf | 5 + .../Properties/xlf/Resources.it.xlf | 5 + .../Properties/xlf/Resources.ja.xlf | 5 + .../Properties/xlf/Resources.ko.xlf | 5 + .../Properties/xlf/Resources.pl.xlf | 5 + .../Properties/xlf/Resources.pt-BR.xlf | 5 + .../Properties/xlf/Resources.ru.xlf | 5 + .../Properties/xlf/Resources.tr.xlf | 5 + .../Properties/xlf/Resources.zh-Hans.xlf | 5 + .../Properties/xlf/Resources.zh-Hant.xlf | 5 + .../Tasks/GenerateJavaStubs.cs | 170 ++--- .../Tasks/GeneratePackageManagerJava.cs | 13 +- .../Tasks/PrepareAbiItems.cs | 54 +- .../Tasks/RemoveRegisterAttribute.cs | 8 +- .../IncrementalBuildTest.cs | 8 +- .../Utilities/EnvironmentHelper.cs | 29 +- .../ARMNativeAssemblerTargetProvider.cs | 6 + ...pplicationConfigNativeAssemblyGenerator.cs | 27 +- .../Utilities/MonoAndroidHelper.cs | 17 +- .../NativeAssemblerTargetProvider.cs | 20 +- .../Utilities/NativeAssemblyDataStream.cs | 235 ------- .../Utilities/NativeAssemblyGenerator.cs | 154 ++++- .../Utilities/NativeTypeMappingData.cs | 60 ++ .../Utilities/TypeMapGenerator.cs | 423 ++++++++++++ .../TypeMappingNativeAssemblyGenerator.cs | 264 +++++-- .../X86NativeAssemblerTargetProvider.cs | 6 + .../Xamarin.Android.Common.targets | 37 +- src/monodroid/jni/android-system.cc | 23 +- src/monodroid/jni/android-system.hh | 3 +- src/monodroid/jni/application_dso_stub.cc | 14 +- src/monodroid/jni/cpp-util.hh | 11 - src/monodroid/jni/embedded-assemblies-zip.cc | 13 - src/monodroid/jni/embedded-assemblies.cc | 649 ++++++++++++++---- src/monodroid/jni/embedded-assemblies.hh | 30 +- src/monodroid/jni/external-api.cc | 10 +- src/monodroid/jni/monodroid-glue-internal.hh | 1 + src/monodroid/jni/monodroid-glue.cc | 24 +- src/monodroid/jni/xamarin-app.h | 42 -- src/monodroid/jni/xamarin-app.hh | 86 +++ .../Tests/InstantRunTest.cs | 7 +- .../Mono.Android-TestsAppBundle.csproj | 2 + .../Mono.Android-TestsMultiDex.csproj | 2 + .../Environment.txt | 7 +- .../MSBuildDeviceIntegration.csv | 6 +- 63 files changed, 1939 insertions(+), 860 deletions(-) create mode 100644 Documentation/guides/messages/xa4308.md delete mode 100644 src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs delete mode 100644 src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs delete mode 100644 src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs create mode 100644 src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs create mode 100644 src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs delete mode 100644 src/monodroid/jni/xamarin-app.h create mode 100644 src/monodroid/jni/xamarin-app.hh diff --git a/Documentation/guides/messages/README.md b/Documentation/guides/messages/README.md index e5a2d79770b..c5aca1639a9 100644 --- a/Documentation/guides/messages/README.md +++ b/Documentation/guides/messages/README.md @@ -104,6 +104,7 @@ ms.date: 01/24/2020 + [XA4305](xa4305.md): MultiDex is enabled, but '{nameof (MultiDexMainDexListFile)}' was not specified. + [XA4306](xa4306.md): R8 does not support \`@(MultiDexMainDexList)\` files when android:minSdkVersion >= 21 + [XA4307](xa4307.md): Invalid ProGuard configuration file. ++ [XA4308](xa4308.md): Failed to generate type maps ## XA5xxx: GCC and toolchain diff --git a/Documentation/guides/messages/xa4308.md b/Documentation/guides/messages/xa4308.md new file mode 100644 index 00000000000..d1f474665e3 --- /dev/null +++ b/Documentation/guides/messages/xa4308.md @@ -0,0 +1,18 @@ +--- +title: Xamarin.Android error XA4308 +description: XA4308 error code +ms.date: 02/07/2020 +--- +# Xamarin.Android error XA4308 + +## Issue + +The `GenerateJavaStubs` task was unable to generate type maps. Detailed diagnostic will be found before this +error in the build log. + +## Solution + +Consider submitting a [bug][bug] if you are getting this warning under +normal circumstances. + +[bug]: https://github.com/xamarin/xamarin-android/wiki/Submitting-Bugs,-Feature-Requests,-and-Pull-Requests diff --git a/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs b/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs index c5bf663aa79..0598e2387f3 100644 --- a/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs +++ b/build-tools/xaprepare/xaprepare/Steps/Step_DownloadMonoArchive.cs @@ -60,26 +60,39 @@ async Task DownloadMonoArchive (Context context) localPath = Path.Combine (context.Properties.GetRequiredValue (KnownProperties.AndroidToolchainCacheDirectory), archiveFileName); } - bool result = await DownloadAndUpackIfNeeded ( - context, - "Mono", - context.MonoArchiveCustomUrl, - localPath, - archiveFileName, - Configurables.Paths.MonoSDKSOutputDir - ); + bool result = false; + for (uint i = 0; i < 3; i++) { + result = await DownloadAndUpackIfNeeded ( + context, + "Mono", + context.MonoArchiveCustomUrl, + localPath, + archiveFileName, + Configurables.Paths.MonoSDKSOutputDir + ); + + if (result) + break; + } if (!result) return false; - return await DownloadAndUpackIfNeeded ( - context, - "Windows Mono", - customUrl: null, - localPath: Configurables.Paths.MonoArchiveWindowsLocalPath, - archiveFileName: Configurables.Paths.MonoArchiveWindowsFileName, - destinationDirectory: Configurables.Paths.BCLWindowsOutputDir - ); + for (uint i = 0; i < 3; i++) { + result = await DownloadAndUpackIfNeeded ( + context, + "Windows Mono", + customUrl: null, + localPath: Configurables.Paths.MonoArchiveWindowsLocalPath, + archiveFileName: Configurables.Paths.MonoArchiveWindowsFileName, + destinationDirectory: Configurables.Paths.BCLWindowsOutputDir + ); + + if (result) + break; + } + + return result; } async Task DownloadAndUpackIfNeeded (Context context, string name, string customUrl, string localPath, string archiveFileName, string destinationDirectory) @@ -109,7 +122,7 @@ async Task DownloadAndUpackIfNeeded (Context context, string name, string await Download (context, url, localPath, $"{name} Archive", archiveFileName, downloadStatus); if (!File.Exists (localPath)) { - Log.InfoLine ($"Download of {name} archive from {url} failed, Mono will be rebuilt"); + Log.InfoLine ($"Download of {name} archive from {url} failed"); return false; } } @@ -118,7 +131,8 @@ async Task DownloadAndUpackIfNeeded (Context context, string name, string if (!await Utilities.Unpack (localPath, tempDir, cleanDestinatioBeforeUnpacking: true)) { Utilities.DeleteFileSilent (localPath); Utilities.DeleteDirectorySilent (destinationDirectory); - Log.WarningLine ($"Failed to unpack {name} archive {localPath}, Mono will be rebuilt"); + Log.WarningLine ($"Failed to unpack {name} archive {localPath}"); + Utilities.DeleteFileSilent (localPath); return false; } diff --git a/src/Mono.Android/Android.Runtime/AndroidRuntime.cs b/src/Mono.Android/Android.Runtime/AndroidRuntime.cs index 478d72dd581..f23cf9f5358 100644 --- a/src/Mono.Android/Android.Runtime/AndroidRuntime.cs +++ b/src/Mono.Android/Android.Runtime/AndroidRuntime.cs @@ -231,9 +231,9 @@ protected override IEnumerable GetTypesForSimpleReference (string jniSimpl protected override string GetSimpleReference (Type type) { - var j = JNIEnv.monodroid_typemap_managed_to_java (type.FullName + ", " + type.Assembly.GetName ().Name); - if (j != IntPtr.Zero) { - return Marshal.PtrToStringAnsi (j); + string j = JNIEnv.TypemapManagedToJava (type); + if (j != null) { + return j; } if (JNIEnv.IsRunningOnDesktop) { return JavaNativeTypeManager.ToJniName (type); @@ -243,9 +243,9 @@ protected override string GetSimpleReference (Type type) protected override IEnumerable GetSimpleReferences (Type type) { - var j = JNIEnv.monodroid_typemap_managed_to_java (type.FullName + ", " + type.Assembly.GetName ().Name); - if (j != IntPtr.Zero) { - yield return Marshal.PtrToStringAnsi (j); + string j = JNIEnv.TypemapManagedToJava (type); + if (j != null) { + yield return j; } if (JNIEnv.IsRunningOnDesktop) { yield return JavaNativeTypeManager.ToJniName (type); diff --git a/src/Mono.Android/Android.Runtime/JNIEnv.cs b/src/Mono.Android/Android.Runtime/JNIEnv.cs index 842837271e7..4b3bc8d3bf8 100644 --- a/src/Mono.Android/Android.Runtime/JNIEnv.cs +++ b/src/Mono.Android/Android.Runtime/JNIEnv.cs @@ -57,10 +57,14 @@ public static partial class JNIEnv { static UncaughtExceptionHandler defaultUncaughtExceptionHandler; internal static bool IsRunningOnDesktop; + internal static bool LogTypemapMissStackTrace; static AndroidRuntime androidRuntime; static BoundExceptionType BoundExceptionType; + [ThreadStatic] + static byte[] mvid_bytes; + internal static AndroidValueManager AndroidValueManager; [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] @@ -146,6 +150,8 @@ internal static unsafe void Initialize (JnienvInitializeArgs* args) partial_timing_sequence = monodroid_timing_start (null); } + LogTypemapMissStackTrace = (args->logCategories & (uint)LogCategories.Assembly) != 0; + gref_gc_threshold = args->grefGcThreshold; java_vm = args->javaVm; @@ -632,16 +638,55 @@ public static string GetClassNameFromInstance (IntPtr jobject) } [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - internal static extern IntPtr monodroid_typemap_managed_to_java (string managed); + static extern IntPtr monodroid_typemap_managed_to_java (byte[] mvid, int token); + + internal static void LogTypemapTrace (StackTrace st) + { + string trace = st.ToString ()?.Trim (); + if (String.IsNullOrEmpty (trace)) + return; + + monodroid_log (LogLevel.Warn, LogCategories.Assembly, "typemap: called from"); + foreach (string line in trace.Split ('\n')) { + monodroid_log (LogLevel.Warn, LogCategories.Assembly, line); + } + } + + internal static string TypemapManagedToJava (Type type) + { + if (mvid_bytes == null) + mvid_bytes = new byte[16]; + + Span mvid = new Span(mvid_bytes); + byte[] mvid_slow = null; + if (!type.Module.ModuleVersionId.TryWriteBytes (mvid)) { + monodroid_log (LogLevel.Warn, LogCategories.Default, $"Failed to obtain module MVID using the fast method, falling back to the slow one"); + mvid_slow = type.Module.ModuleVersionId.ToByteArray (); + } + + IntPtr ret = monodroid_typemap_managed_to_java (mvid_slow == null ? mvid_bytes : mvid_slow, type.MetadataToken); + + if (ret == IntPtr.Zero) { + if (LogTypemapMissStackTrace) { + monodroid_log (LogLevel.Warn, LogCategories.Default, $"typemap: failed to map managed type to Java type: {type.AssemblyQualifiedName} (Module ID: {type.Module.ModuleVersionId}; Type token: {type.MetadataToken})"); + LogTypemapTrace (new StackTrace (true)); + } + + return null; + } + + return Marshal.PtrToStringAnsi (ret); + } public static string GetJniName (Type type) { if (type == null) throw new ArgumentNullException ("type"); - var java = monodroid_typemap_managed_to_java (type.FullName + ", " + type.Assembly.GetName ().Name); - return java == IntPtr.Zero + + string java = TypemapManagedToJava (type); + return java == null ? JavaNativeTypeManager.ToJniName (type) - : Marshal.PtrToStringAnsi (java); + : java; } public static IntPtr ToJniHandle (IJavaObject value) diff --git a/src/Mono.Android/Java.Interop/TypeManager.cs b/src/Mono.Android/Java.Interop/TypeManager.cs index 0cf4265f62c..ebab6f33f49 100644 --- a/src/Mono.Android/Java.Interop/TypeManager.cs +++ b/src/Mono.Android/Java.Interop/TypeManager.cs @@ -2,6 +2,7 @@ using System.Collections.Generic; using System.Reflection; using System.Linq; +using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using Java.Interop.Tools.TypeNameMappings; @@ -203,22 +204,25 @@ static Exception CreateJavaLocationException () return new JavaLocationException (loc.ToString ()); } - [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - internal static extern IntPtr monodroid_typemap_java_to_managed (string java); + [MethodImplAttribute(MethodImplOptions.InternalCall)] + static extern Type monodroid_typemap_java_to_managed (string java_type_name); internal static Type GetJavaToManagedType (string class_name) { - var t = monodroid_typemap_java_to_managed (class_name); - if (t != IntPtr.Zero) - return Type.GetType (Marshal.PtrToStringAnsi (t)); + Type type = monodroid_typemap_java_to_managed (class_name); + if (type != null) + return type; if (!JNIEnv.IsRunningOnDesktop) { + // Miss message is logged in the native runtime + if (JNIEnv.LogTypemapMissStackTrace) + JNIEnv.LogTypemapTrace (new System.Diagnostics.StackTrace (true)); return null; } __TypeRegistrations.RegisterPackages (); - var type = (Type) null; + type = null; int ls = class_name.LastIndexOf ('/'); var package = ls >= 0 ? class_name.Substring (0, ls) : ""; List> mappers; diff --git a/src/Mono.Android/Properties/AssemblyInfo.cs.in b/src/Mono.Android/Properties/AssemblyInfo.cs.in index 3063db7a1b8..960d373e6fa 100644 --- a/src/Mono.Android/Properties/AssemblyInfo.cs.in +++ b/src/Mono.Android/Properties/AssemblyInfo.cs.in @@ -27,3 +27,7 @@ using System.Runtime.CompilerServices; [assembly: System.Runtime.CompilerServices.TypeForwardedToAttribute(typeof(System.Drawing.Size))] [assembly: System.Runtime.CompilerServices.TypeForwardedToAttribute(typeof(System.Drawing.SizeF))] [assembly: System.Runtime.CompilerServices.TypeForwardedToAttribute(typeof(System.Drawing.SystemColors))] +[assembly: InternalsVisibleTo("Mono.Android-Tests, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] +[assembly: InternalsVisibleTo("Java.Interop-Tests, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] +[assembly: InternalsVisibleTo("Mono.Android-TestsMultiDex, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] +[assembly: InternalsVisibleTo("Mono.Android-TestsAppBundle, PublicKey=0024000004800000940000000602000000240000525341310004000011000000438ac2a5acfbf16cbd2b2b47a62762f273df9cb2795ceccdf77d10bf508e69e7a362ea7a45455bbf3ac955e1f2e2814f144e5d817efc4c6502cc012df310783348304e3ae38573c6d658c234025821fda87a0be8a0d504df564e2c93b2b878925f42503e9d54dfef9f9586d9e6f38a305769587b1de01f6c0410328b2c9733db")] diff --git a/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj b/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj index a587304a775..fe28f678c9a 100644 --- a/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj +++ b/src/Mono.Android/Test/Java.Interop-Tests/Java.Interop-Tests.csproj @@ -17,6 +17,8 @@ Off v10.0 true + true + ..\..\..\..\product.snk @@ -48,12 +50,11 @@ - ..\..\..\..\packages\mono.linq.expressions.2.0.0\lib\netstandard2.0\Mono.Linq.Expressions.dll + ..\..\..\..\packages\Mono.Linq.Expressions.2.0.0\lib\netstandard2.0\Mono.Linq.Expressions.dll - diff --git a/src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs b/src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs deleted file mode 100644 index 887802784b2..00000000000 --- a/src/Mono.Android/Test/Java.Interop-Tests/Properties/AssemblyInfo.cs +++ /dev/null @@ -1,30 +0,0 @@ -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using Android.App; - -// General Information about an assembly is controlled through the following -// set of attributes. Change these attribute values to modify the information -// associated with an assembly. -[assembly: AssemblyTitle("Java.Interop_Tests")] -[assembly: AssemblyDescription("")] -[assembly: AssemblyConfiguration("")] -[assembly: AssemblyCompany("")] -[assembly: AssemblyProduct("Java.Interop_Tests")] -[assembly: AssemblyCopyright("Copyright © 2018")] -[assembly: AssemblyTrademark("")] -[assembly: AssemblyCulture("")] -[assembly: ComVisible(false)] - -// Version information for an assembly consists of the following four values: -// -// Major Version -// Minor Version -// Build Number -// Revision -// -// You can specify all the values or you can default the Build and Revision Numbers -// by using the '*' as shown below: -// [assembly: AssemblyVersion("1.0.*")] -[assembly: AssemblyVersion("1.0.0.0")] -[assembly: AssemblyFileVersion("1.0.0.0")] diff --git a/src/Mono.Android/Test/Java.Interop/JnienvTest.cs b/src/Mono.Android/Test/Java.Interop/JnienvTest.cs index a4f90046cba..d6b54c45c97 100644 --- a/src/Mono.Android/Test/Java.Interop/JnienvTest.cs +++ b/src/Mono.Android/Test/Java.Interop/JnienvTest.cs @@ -1,5 +1,6 @@ using System; using System.Linq; +using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using System.Threading; @@ -373,32 +374,27 @@ public void MoarThreadingTests () Assert.IsNull (ignore_t2, string.Format ("No exception should be thrown [t2]! Got: {0}", ignore_t2)); } - [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - static extern IntPtr monodroid_typemap_java_to_managed (string java); - [Test] public void JavaToManagedTypeMapping () { - var m = monodroid_typemap_java_to_managed ("android/content/res/Resources"); - Assert.AreNotEqual (IntPtr.Zero, m); - m = monodroid_typemap_java_to_managed ("this/type/does/not/exist"); - Assert.AreEqual (IntPtr.Zero, m); + Type m = Java.Interop.TypeManager.GetJavaToManagedType ("android/content/res/Resources"); + Assert.AreNotEqual (null, m); + m = Java.Interop.TypeManager.GetJavaToManagedType ("this/type/does/not/exist"); + Assert.AreEqual (null, m); } [DllImport ("__Internal", CallingConvention = CallingConvention.Cdecl)] - static extern IntPtr monodroid_typemap_managed_to_java (string java); - - string GetTypeName (Type type) - { - return type.FullName + ", " + type.Assembly.GetName ().Name; - } + static extern IntPtr monodroid_typemap_managed_to_java (byte[] mvid, int token); [Test] public void ManagedToJavaTypeMapping () { - var m = monodroid_typemap_managed_to_java (GetTypeName (typeof (Activity))); + Type type = typeof(Activity); + var m = monodroid_typemap_managed_to_java (type.Module.ModuleVersionId.ToByteArray (), type.MetadataToken); Assert.AreNotEqual (IntPtr.Zero, m, "`Activity` subclasses Java.Lang.Object, it should be in the typemap!"); - m = monodroid_typemap_managed_to_java (GetTypeName (typeof (JnienvTest))); + + type = typeof (JnienvTest); + m = monodroid_typemap_managed_to_java (type.Module.ModuleVersionId.ToByteArray (), type.MetadataToken); Assert.AreEqual (IntPtr.Zero, m, "`JnienvTest` does *not* subclass Java.Lang.Object, it should *not* be in the typemap!"); } diff --git a/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj b/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj index 80a77968eac..98d84f6debf 100644 --- a/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj +++ b/src/Mono.Android/Test/Mono.Android-Test.Library/Mono.Android-Test.Library.csproj @@ -46,7 +46,6 @@ - diff --git a/src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs b/src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs deleted file mode 100644 index ea0d06d7073..00000000000 --- a/src/Mono.Android/Test/Mono.Android-Test.Library/Properties/AssemblyInfo.cs +++ /dev/null @@ -1,30 +0,0 @@ -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using Android.App; - -// General Information about an assembly is controlled through the following -// set of attributes. Change these attribute values to modify the information -// associated with an assembly. -[assembly: AssemblyTitle("Mono.Android_Test.Library")] -[assembly: AssemblyDescription("")] -[assembly: AssemblyConfiguration("")] -[assembly: AssemblyCompany("")] -[assembly: AssemblyProduct("Mono.Android_Test.Library")] -[assembly: AssemblyCopyright("Copyright © 2018")] -[assembly: AssemblyTrademark("")] -[assembly: AssemblyCulture("")] -[assembly: ComVisible(false)] - -// Version information for an assembly consists of the following four values: -// -// Major Version -// Minor Version -// Build Number -// Revision -// -// You can specify all the values or you can default the Build and Revision Numbers -// by using the '*' as shown below: -// [assembly: AssemblyVersion("1.0.*")] -[assembly: AssemblyVersion("1.0.0.0")] -[assembly: AssemblyFileVersion("1.0.0.0")] diff --git a/src/Mono.Android/Test/Mono.Android-Tests.csproj b/src/Mono.Android/Test/Mono.Android-Tests.csproj index 96118c34a18..76e6a21f8fc 100644 --- a/src/Mono.Android/Test/Mono.Android-Tests.csproj +++ b/src/Mono.Android/Test/Mono.Android-Tests.csproj @@ -18,6 +18,8 @@ Properties\AndroidManifest.xml armeabi-v7a;x86 False + true + ..\..\..\product.snk v10.0 d8 diff --git a/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets b/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets index 3f4ce816da6..2a5ee84f13a 100644 --- a/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets +++ b/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.Designer.targets @@ -56,8 +56,8 @@ Copyright (C) 2016 Xamarin. All rights reserved. Include="@(ResolvedUserAssemblies->'$(MonoAndroidIntermediateAssemblyDir)%(Filename)%(Extension)')" /> <_ResolvedUserMonoAndroidAssembliesForDesigner - Include="@(ResolvedUserAssemblies)" - Condition="'%(ResolvedUserAssemblies.TargetFrameworkIdentifier)' == 'MonoAndroid' Or '%(ResolvedUserAssemblies.HasMonoAndroidReference)' == 'True'" + Include="@(_ResolvedUserAssemblies)" + Condition="'%(_ResolvedUserAssemblies.TargetFrameworkIdentifier)' == 'MonoAndroid' Or '%(_ResolvedUserAssemblies.HasMonoAndroidReference)' == 'True'" /> @@ -86,6 +86,8 @@ Copyright (C) 2016 Xamarin. All rights reserved. AndroidSdkDir="$(_AndroidSdkDirectory)" PackageName="$(_AndroidPackage)" OutputDirectory="$(IntermediateOutputPath)android" + TypemapOutputDirectory="$(_NativeAssemblySourceDir)" + GenerateNativeAssembly="false" MergedAndroidManifestOutput="$(_ManifestOutput)" UseSharedRuntime="$(AndroidUseSharedRuntime)" EmbedAssemblies="$(EmbedAssembliesIntoApk)" @@ -94,7 +96,8 @@ Copyright (C) 2016 Xamarin. All rights reserved. ApplicationJavaClass="$(AndroidApplicationJavaClass)" FrameworkDirectories="$(_XATargetFrameworkDirectories);$(_XATargetFrameworkDirectories)Facades" AcwMapFile="$(_AcwMapFile)" - SupportedAbis="$(_BuildTargetAbis)"> + SupportedAbis="$(_BuildTargetAbis)" + InstantRunEnabled="$(_InstantRunEnabled)"> + EnablePreloadAssembliesDefault="$(_AndroidEnablePreloadAssembliesDefault)" + InstantRunEnabled="$(_InstantRunEnabled)"> diff --git a/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs b/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs index 144836d16cd..cfd6318ad96 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs +++ b/src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs @@ -303,6 +303,14 @@ internal static string XA4305_File_Missing { } } + /// Looks up a localized string similar to Failed to generate type maps. + /// + internal static string XA4308 { + get { + return ResourceManager.GetString("XA4308", resourceCulture); + } + } + /// /// Looks up a localized string similar to Missing Android NDK toolchains directory '{0}'. Please install the Android NDK.. /// diff --git a/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx b/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx index 9c203bb59e4..8c54034b470 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx +++ b/src/Xamarin.Android.Build.Tasks/Properties/Resources.resx @@ -238,6 +238,9 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. {0} - The path of the missing directory @@ -268,4 +271,4 @@ {0} - The managed type name {1} - The exception message of the associated exception - \ No newline at end of file + diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf index c81c114f8c2..c748de2638d 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.cs.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf index 5f7c6618d2d..570747d4d2f 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.de.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf index dd6a13df338..dcb7dc84679 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.es.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf index a4b1eadf814..83fad9636cf 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.fr.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf index 35986cc9035..4ab899e99d1 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.it.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf index 2e150b819a4..1557560da43 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ja.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf index bdc43885ac8..c987644bb4f 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ko.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf index 0a7f3f1c707..a2874bbf844 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pl.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf index f7251109ee8..f513ad17b34 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.pt-BR.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf index 1f0229a480d..f7225aa2f42 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.ru.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf index c5c030af400..5c66871f3b3 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.tr.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf index 01293297d69..2188eb2d0d5 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hans.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf index 6984cbcc842..c4a92aa9f98 100644 --- a/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf +++ b/src/Xamarin.Android.Build.Tasks/Properties/xlf/Resources.zh-Hant.xlf @@ -154,6 +154,11 @@ MultiDex is enabled, but main dex list file '{0}' does not exist. The following are literal names and should not be translated: MultiDex + + Failed to generate type maps + Failed to generate type maps + + Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. Missing Android NDK toolchains directory '{0}'. Please install the Android NDK. diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs b/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs index fcc277569f5..5664b18aaf5 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs @@ -42,6 +42,12 @@ public class GenerateJavaStubs : AndroidTask [Required] public string [] SupportedAbis { get; set; } + [Required] + public string TypemapOutputDirectory { get; set; } + + [Required] + public bool GenerateNativeAssembly { get; set; } + public string ManifestTemplate { get; set; } public string[] MergedManifestDocuments { get; set; } @@ -71,6 +77,9 @@ public class GenerateJavaStubs : AndroidTask public string ApplicationJavaClass { get; set; } + [Output] + public string [] GeneratedBinaryTypeMaps { get; set; } + internal const string AndroidSkipJavaStubGeneration = "AndroidSkipJavaStubGeneration"; public override bool RunTask () @@ -92,8 +101,6 @@ public override bool RunTask () // by ensuring that the target outputs have been deleted. Files.DeleteFile (MergedAndroidManifestOutput, Log); Files.DeleteFile (AcwMapFile, Log); - Files.DeleteFile (Path.Combine (OutputDirectory, "typemap.jm"), Log); - Files.DeleteFile (Path.Combine (OutputDirectory, "typemap.mj"), Log); } return !Log.HasLoggedErrors; @@ -110,73 +117,95 @@ void Run (DirectoryAssemblyResolver res) } // Put every assembly we'll need in the resolver + bool hasExportReference = false; + bool haveMonoAndroid = false; + var allTypemapAssemblies = new HashSet (StringComparer.OrdinalIgnoreCase); + var userAssemblies = new Dictionary (StringComparer.OrdinalIgnoreCase); foreach (var assembly in ResolvedAssemblies) { - if (bool.TryParse (assembly.GetMetadata (AndroidSkipJavaStubGeneration), out bool value) && value) { + bool value; + if (bool.TryParse (assembly.GetMetadata (AndroidSkipJavaStubGeneration), out value) && value) { Log.LogDebugMessage ($"Skipping Java Stub Generation for {assembly.ItemSpec}"); continue; } + + bool addAssembly = false; + string fileName = Path.GetFileName (assembly.ItemSpec); + if (!hasExportReference && String.Compare ("Mono.Android.Export.dll", fileName, StringComparison.OrdinalIgnoreCase) == 0) { + hasExportReference = true; + addAssembly = true; + } else if (!haveMonoAndroid && String.Compare ("Mono.Android.dll", fileName, StringComparison.OrdinalIgnoreCase) == 0) { + haveMonoAndroid = true; + addAssembly = true; + } else if (MonoAndroidHelper.FrameworkAssembliesToTreatAsUserAssemblies.Contains (fileName)) { + if (!bool.TryParse (assembly.GetMetadata (AndroidSkipJavaStubGeneration), out value) || !value) { + string name = Path.GetFileNameWithoutExtension (fileName); + if (!userAssemblies.ContainsKey (name)) + userAssemblies.Add (name, assembly.ItemSpec); + addAssembly = true; + } + } + + if (addAssembly) { + allTypemapAssemblies.Add (assembly.ItemSpec); + } + res.Load (assembly.ItemSpec); } - // However we only want to look for JLO types in user code - List assemblies = new List (); + // However we only want to look for JLO types in user code for Java stub code generation foreach (var asm in ResolvedUserAssemblies) { if (bool.TryParse (asm.GetMetadata (AndroidSkipJavaStubGeneration), out bool value) && value) { Log.LogDebugMessage ($"Skipping Java Stub Generation for {asm.ItemSpec}"); continue; } - if (!assemblies.All (x => Path.GetFileName (x) != Path.GetFileName (asm.ItemSpec))) - continue; - Log.LogDebugMessage ($"Adding {asm.ItemSpec} to assemblies."); - assemblies.Add (asm.ItemSpec); - } - foreach (var asm in MonoAndroidHelper.GetFrameworkAssembliesToTreatAsUserAssemblies (ResolvedAssemblies)) { - if (bool.TryParse (asm.GetMetadata (AndroidSkipJavaStubGeneration), out bool value) && value) { - Log.LogDebugMessage ($"Skipping Java Stub Generation for {asm.ItemSpec}"); - continue; - } - if (!assemblies.All (x => Path.GetFileName (x) != Path.GetFileName (asm.ItemSpec))) - continue; - Log.LogDebugMessage ($"Adding {asm.ItemSpec} to assemblies."); - assemblies.Add (asm.ItemSpec); + allTypemapAssemblies.Add (asm.ItemSpec); + userAssemblies.Add (Path.GetFileNameWithoutExtension (asm.ItemSpec), asm.ItemSpec); } // Step 1 - Find all the JLO types var scanner = new JavaTypeScanner (this.CreateTaskLogger ()) { ErrorOnCustomJavaObject = ErrorOnCustomJavaObject, }; - var all_java_types = scanner.GetJavaTypes (assemblies, res); - WriteTypeMappings (all_java_types); + List allJavaTypes = scanner.GetJavaTypes (allTypemapAssemblies, res); - var java_types = all_java_types - .Where (t => !JavaTypeScanner.ShouldSkipJavaCallableWrapperGeneration (t)) - .ToArray (); + // Step 2 - Generate type maps + // Type mappings need to use all the assemblies, always. + WriteTypeMappings (res, allJavaTypes); - // Step 2 - Generate Java stub code + var javaTypes = new List (); + foreach (TypeDefinition td in allJavaTypes) { + if (!userAssemblies.ContainsKey (td.Module.Assembly.Name.Name) || JavaTypeScanner.ShouldSkipJavaCallableWrapperGeneration (td)) { + continue; + } + + javaTypes.Add (td); + } + + // Step 3 - Generate Java stub code var success = Generator.CreateJavaSources ( Log, - java_types, + javaTypes, Path.Combine (OutputDirectory, "src"), ApplicationJavaClass, AndroidSdkPlatform, UseSharedRuntime, int.Parse (AndroidSdkPlatform) <= 10, - ResolvedAssemblies.Any (assembly => Path.GetFileName (assembly.ItemSpec) == "Mono.Android.Export.dll")); + hasExportReference); if (!success) return; // We need to save a map of .NET type -> ACW type for resource file fixups - var managed = new Dictionary (java_types.Length, StringComparer.Ordinal); - var java = new Dictionary (java_types.Length, StringComparer.Ordinal); + var managed = new Dictionary (javaTypes.Count, StringComparer.Ordinal); + var java = new Dictionary (javaTypes.Count, StringComparer.Ordinal); var managedConflicts = new Dictionary> (0, StringComparer.Ordinal); var javaConflicts = new Dictionary> (0, StringComparer.Ordinal); // Allocate a MemoryStream with a reasonable guess at its capacity - using (var stream = new MemoryStream (java_types.Length * 32)) + using (var stream = new MemoryStream (javaTypes.Count * 32)) using (var acw_map = new StreamWriter (stream)) { - foreach (var type in java_types) { + foreach (TypeDefinition type in javaTypes) { string managedKey = type.FullName.Replace ('/', '.'); string javaKey = JavaNativeTypeManager.ToJniName (type).Replace ('/', '.'); @@ -241,7 +270,7 @@ void Run (DirectoryAssemblyResolver res) manifest.PackageName = PackageName; manifest.ApplicationName = ApplicationName ?? PackageName; manifest.Placeholders = ManifestPlaceholders; - manifest.Assemblies.AddRange (assemblies); + manifest.Assemblies.AddRange (userAssemblies.Values); manifest.Resolver = res; manifest.SdkDir = AndroidSdkDir; manifest.SdkVersion = AndroidSdkPlatform; @@ -250,7 +279,7 @@ void Run (DirectoryAssemblyResolver res) manifest.NeedsInternet = NeedsInternet; manifest.InstantRunEnabled = InstantRunEnabled; - var additionalProviders = manifest.Merge (Log, all_java_types, ApplicationJavaClass, EmbedAssemblies, BundledWearApplicationName, MergedManifestDocuments); + var additionalProviders = manifest.Merge (Log, allJavaTypes, ApplicationJavaClass, EmbedAssemblies, BundledWearApplicationName, MergedManifestDocuments); using (var stream = new MemoryStream ()) { manifest.Save (Log, stream); @@ -272,7 +301,7 @@ void Run (DirectoryAssemblyResolver res) // Create additional application java sources. StringWriter regCallsWriter = new StringWriter (); regCallsWriter.WriteLine ("\t\t// Application and Instrumentation ACWs must be registered first."); - foreach (var type in java_types) { + foreach (var type in javaTypes) { if (JavaNativeTypeManager.IsApplication (type) || JavaNativeTypeManager.IsInstrumentation (type)) { string javaKey = JavaNativeTypeManager.ToJniName (type).Replace ('/', '.'); regCallsWriter.WriteLine ("\t\tmono.android.Runtime.register (\"{0}\", {1}.class, {1}.__md_methods);", @@ -301,75 +330,12 @@ void SaveResource (string resource, string filename, string destDir, Func types) - { - void logger (TraceLevel level, string value) => Log.LogDebugMessage (value); - TypeNameMapGenerator createTypeMapGenerator () => UseSharedRuntime ? - new TypeNameMapGenerator (types, logger) : - new TypeNameMapGenerator (ResolvedAssemblies.Select (p => p.ItemSpec), logger); - using (var gen = createTypeMapGenerator ()) { - using (var ms = new MemoryStream ()) { - UpdateWhenChanged (Path.Combine (OutputDirectory, "typemap.jm"), "jm", ms, gen.WriteJavaToManaged); - UpdateWhenChanged (Path.Combine (OutputDirectory, "typemap.mj"), "mj", ms, gen.WriteManagedToJava); - } - } - } - - void UpdateWhenChanged (string path, string type, MemoryStream ms, Action generator) + void WriteTypeMappings (DirectoryAssemblyResolver resolver, List types) { - if (!EmbedAssemblies) { - ms.SetLength (0); - generator (ms); - MonoAndroidHelper.CopyIfStreamChanged (ms, path); - } - - string dataFilePath = $"{path}.inc"; - using (var stream = new NativeAssemblyDataStream ()) { - if (EmbedAssemblies) { - generator (stream); - stream.EndOfFile (); - MonoAndroidHelper.CopyIfStreamChanged (stream, dataFilePath); - } else { - stream.EmptyFile (); - } - - var generatedFiles = new List (); - string mappingFieldName = $"{type}_typemap"; - string dataFileName = Path.GetFileName (dataFilePath); - NativeAssemblerTargetProvider asmTargetProvider; - var utf8Encoding = new UTF8Encoding (false); - foreach (string abi in SupportedAbis) { - ms.SetLength (0); - switch (abi.Trim ()) { - case "armeabi-v7a": - asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: false); - break; - - case "arm64-v8a": - asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: true); - break; - - case "x86": - asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: false); - break; - - case "x86_64": - asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: true); - break; - - default: - throw new InvalidOperationException ($"Unknown ABI {abi}"); - } - - var asmgen = new TypeMappingNativeAssemblyGenerator (asmTargetProvider, stream, dataFileName, stream.MapByteCount, mappingFieldName); - asmgen.EmbedAssemblies = EmbedAssemblies; - string asmFileName = $"{path}.{abi.Trim ()}.s"; - using (var sw = new StreamWriter (ms, utf8Encoding, bufferSize: 8192, leaveOpen: true)) { - asmgen.Write (sw, dataFileName); - MonoAndroidHelper.CopyIfStreamChanged (ms, asmFileName); - } - } - } + var tmg = new TypeMapGenerator ((string message) => Log.LogDebugMessage (message), SupportedAbis); + if (!tmg.Generate (types, TypemapOutputDirectory, GenerateNativeAssembly)) + throw new XamarinAndroidException (4308, Properties.Resources.XA4308); + GeneratedBinaryTypeMaps = tmg.GeneratedBinaryTypeMaps.ToArray (); } } } diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs b/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs index 45a59759110..2764f083cc2 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/GeneratePackageManagerJava.cs @@ -58,6 +58,9 @@ public class GeneratePackageManagerJava : AndroidTask [Required] public bool EnablePreloadAssembliesDefault { get; set; } + [Required] + public bool InstantRunEnabled { get; set; } + public string BoundExceptionType { get; set; } public string PackageNamingPolicy { get; set; } @@ -252,7 +255,8 @@ void AddEnvironment () foreach (string abi in SupportedAbis) { ms.SetLength (0); NativeAssemblerTargetProvider asmTargetProvider; - string asmFileName = Path.Combine (EnvironmentOutputDirectory, $"environment.{abi.ToLowerInvariant ()}.s"); + string baseAsmFilePath = Path.Combine (EnvironmentOutputDirectory, $"environment.{abi.ToLowerInvariant ()}"); + string asmFilePath = $"{baseAsmFilePath}.s"; switch (abi.Trim ()) { case "armeabi-v7a": asmTargetProvider = new ARMNativeAssemblerTargetProvider (false); @@ -274,7 +278,7 @@ void AddEnvironment () throw new InvalidOperationException ($"Unknown ABI {abi}"); } - var asmgen = new ApplicationConfigNativeAssemblyGenerator (asmTargetProvider, environmentVariables, systemProperties) { + var asmgen = new ApplicationConfigNativeAssemblyGenerator (asmTargetProvider, baseAsmFilePath, environmentVariables, systemProperties) { IsBundledApp = IsBundledApplication, UsesMonoAOT = usesMonoAOT, UsesMonoLLVM = EnableLLVM, @@ -284,11 +288,12 @@ void AddEnvironment () BrokenExceptionTransitions = brokenExceptionTransitions, PackageNamingPolicy = pnp, BoundExceptionType = boundExceptionType, + InstantRunEnabled = InstantRunEnabled, }; using (var sw = new StreamWriter (ms, utf8Encoding, bufferSize: 8192, leaveOpen: true)) { - asmgen.Write (sw, asmFileName); - MonoAndroidHelper.CopyIfStreamChanged (ms, asmFileName); + asmgen.Write (sw); + MonoAndroidHelper.CopyIfStreamChanged (ms, asmFilePath); } } diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs b/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs index 00517cdbc0b..d298dfd4ce0 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/PrepareAbiItems.cs @@ -1,4 +1,5 @@ using System; +using System.IO; using System.Collections.Generic; using Microsoft.Build.Framework; @@ -8,27 +9,68 @@ namespace Xamarin.Android.Tasks { public class PrepareAbiItems : AndroidTask { + const string ArmV7a = "armeabi-v7a"; + const string TypeMapBase = "typemaps"; + const string EnvBase = "environment"; + public override string TaskPrefix => "PAI"; [Required] public string [] BuildTargetAbis { get; set; } [Required] - public string ItemNamePattern { get; set; } + public string NativeSourcesDir { get; set; } + + [Required] + public bool TypeMapMode { get; set; } [Output] - public ITaskItem[] OutputItems { get; set; } + public ITaskItem[] AssemblySources { get; set; } + + [Output] + public ITaskItem[] AssemblyIncludes { get; set; } public override bool RunTask () { - var items = new List (); + var sources = new List (); + var includes = new List (); + bool haveSharedSource = false; + bool haveArmV7SharedSource = false; + string baseName = TypeMapMode ? TypeMapBase : EnvBase; + TaskItem item; foreach (string abi in BuildTargetAbis) { - var item = new TaskItem (ItemNamePattern.Replace ("@abi@", abi)); + if (TypeMapMode) { + if (String.Compare (ArmV7a, abi, StringComparison.Ordinal) == 0) + haveArmV7SharedSource = true; + else + haveSharedSource = true; + } + + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.{abi}.s")); item.SetMetadata ("abi", abi); - items.Add (item); + sources.Add (item); + + if (!TypeMapMode) + continue; + + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.{abi}-managed.inc")); + item.SetMetadata ("abi", abi); + includes.Add (item); + } + + if (haveArmV7SharedSource) { + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.{ArmV7a}-shared.inc")); + item.SetMetadata ("abi", ArmV7a); + includes.Add (item); + } + + if (haveSharedSource) { + item = new TaskItem (Path.Combine (NativeSourcesDir, $"{baseName}.shared.inc")); + includes.Add (item); } - OutputItems = items.ToArray (); + AssemblySources = sources.ToArray (); + AssemblyIncludes = includes.ToArray (); return !Log.HasLoggedErrors; } } diff --git a/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs b/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs index e2883f77e7c..4163db2e921 100644 --- a/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs +++ b/src/Xamarin.Android.Build.Tasks/Tasks/RemoveRegisterAttribute.cs @@ -19,21 +19,17 @@ public class RemoveRegisterAttribute : AndroidTask [Required] public ITaskItem[] ShrunkFrameworkAssemblies { get; set; } - public bool Deterministic { get; set; } - public override bool RunTask () { // Find Mono.Android.dll var mono_android = ShrunkFrameworkAssemblies.First (f => Path.GetFileNameWithoutExtension (f.ItemSpec) == "Mono.Android").ItemSpec; - var writerParameters = new WriterParameters { - DeterministicMvid = Deterministic, - }; + using (var assembly = AssemblyDefinition.ReadAssembly (mono_android, new ReaderParameters { ReadWrite = true })) { // Strip out [Register] attributes foreach (TypeDefinition type in assembly.MainModule.Types) ProcessType (type); - assembly.Write (writerParameters); + assembly.Write (); } return true; diff --git a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs index cc221f64c11..d790cfbf28e 100644 --- a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs +++ b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs @@ -805,10 +805,10 @@ public void GenerateJavaStubsAndAssembly ([Values (true, false)] bool isRelease) readonly string [] ExpectedAssemblyFiles = new [] { Path.Combine ("android", "environment.armeabi-v7a.o"), Path.Combine ("android", "environment.armeabi-v7a.s"), - Path.Combine ("android", "typemap.mj.armeabi-v7a.o"), - Path.Combine ("android", "typemap.mj.armeabi-v7a.s"), - Path.Combine ("android", "typemap.jm.armeabi-v7a.o"), - Path.Combine ("android", "typemap.jm.armeabi-v7a.s"), + Path.Combine ("android", "typemaps.armeabi-v7a-managed.inc"), + Path.Combine ("android", "typemaps.armeabi-v7a-shared.inc"), + Path.Combine ("android", "typemaps.armeabi-v7a.o"), + Path.Combine ("android", "typemaps.armeabi-v7a.s"), Path.Combine ("app_shared_libraries", "armeabi-v7a", "libxamarin-app.so") }; diff --git a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs index d253dc90264..a81b9aefbe7 100644 --- a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs +++ b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Utilities/EnvironmentHelper.cs @@ -22,17 +22,18 @@ public sealed class ApplicationConfig public bool uses_assembly_preload; public bool is_a_bundled_app; public bool broken_exception_transitions; + public bool instant_run_enabled; public byte bound_stream_io_exception_type; public uint package_naming_policy; public uint environment_variable_count; public uint system_property_count; public string android_package_name; }; - const uint ApplicationConfigFieldCount = 10; + const uint ApplicationConfigFieldCount = 11; static readonly object ndkInitLock = new object (); static readonly char[] readElfFieldSeparator = new [] { ' ', '\t' }; - static readonly Regex stringLabelRegex = new Regex ("^\\.L\\.str\\.[0-9]+:", RegexOptions.Compiled); + static readonly Regex stringLabelRegex = new Regex ("^\\.L\\.env\\.str\\.[0-9]+:", RegexOptions.Compiled); static readonly HashSet expectedPointerTypes = new HashSet (StringComparer.Ordinal) { ".long", @@ -49,10 +50,11 @@ public sealed class ApplicationConfig "app_environment_variables", "app_system_properties", "application_config", - "jm_typemap", - "jm_typemap_header", - "mj_typemap", - "mj_typemap_header", + "map_modules", + "map_module_count", + "java_type_count", + "java_name_width", + "map_java", "mono_aot_mode_name", }; @@ -136,27 +138,32 @@ static ApplicationConfig ReadApplicationConfig (string envFile) ret.broken_exception_transitions = ConvertFieldToBool ("broken_exception_transitions", envFile, i, field [1]); break; - case 5: // bound_stream_io_exception_type: byte / .byte + case 5: // instant_run_enabled: bool / .byte + AssertFieldType (envFile, ".byte", field [0], i); + ret.instant_run_enabled = ConvertFieldToBool ("instant_run_enabled", envFile, i, field [1]); + break; + + case 6: // bound_stream_io_exception_type: byte / .byte AssertFieldType (envFile, ".byte", field [0], i); ret.bound_stream_io_exception_type = ConvertFieldToByte ("bound_stream_io_exception_type", envFile, i, field [1]); break; - case 6: // package_naming_policy: uint32_t / .word | .long + case 7: // package_naming_policy: uint32_t / .word | .long Assert.IsTrue (expectedUInt32Types.Contains (field [0]), $"Unexpected uint32_t field type in '{envFile}:{i}': {field [0]}"); ret.package_naming_policy = ConvertFieldToUInt32 ("package_naming_policy", envFile, i, field [1]); break; - case 7: // environment_variable_count: uint32_t / .word | .long + case 8: // environment_variable_count: uint32_t / .word | .long Assert.IsTrue (expectedUInt32Types.Contains (field [0]), $"Unexpected uint32_t field type in '{envFile}:{i}': {field [0]}"); ret.environment_variable_count = ConvertFieldToUInt32 ("environment_variable_count", envFile, i, field [1]); break; - case 8: // system_property_count: uint32_t / .word | .long + case 9: // system_property_count: uint32_t / .word | .long Assert.IsTrue (expectedUInt32Types.Contains (field [0]), $"Unexpected uint32_t field type in '{envFile}:{i}': {field [0]}"); ret.system_property_count = ConvertFieldToUInt32 ("system_property_count", envFile, i, field [1]); break; - case 9: // android_package_name: string / [pointer type] + case 10: // android_package_name: string / [pointer type] Assert.IsTrue (expectedPointerTypes.Contains (field [0]), $"Unexpected pointer field type in '{envFile}:{i}': {field [0]}"); pointers.Add (field [1].Trim ()); break; diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs b/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs index 2791da006a0..e563f64ffe6 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/ARMNativeAssemblerTargetProvider.cs @@ -5,9 +5,15 @@ namespace Xamarin.Android.Tasks { class ARMNativeAssemblerTargetProvider : NativeAssemblerTargetProvider { + const string ARMV7a = "armeabi-v7a"; + const string ARMV8a = "arm64-v8a"; + public override bool Is64Bit { get; } public override string PointerFieldType { get; } public override string TypePrefix { get; } + public override string AbiName => Is64Bit ? ARMV8a : ARMV7a; + public override uint MapModulesAlignBits => Is64Bit ? 3u : 2u; + public override uint MapJavaAlignBits { get; } = 2; public ARMNativeAssemblerTargetProvider (bool is64Bit) { diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs index 456db8704ba..1c5b70cf0f2 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/ApplicationConfigNativeAssemblyGenerator.cs @@ -20,11 +20,12 @@ class ApplicationConfigNativeAssemblyGenerator : NativeAssemblyGenerator public string AndroidPackageName { get; set; } public bool BrokenExceptionTransitions { get; set; } public global::Android.Runtime.BoundExceptionType BoundExceptionType { get; set; } + public bool InstantRunEnabled { get; set; } public PackageNamingPolicy PackageNamingPolicy { get; set; } - public ApplicationConfigNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, IDictionary environmentVariables, IDictionary systemProperties) - : base (targetProvider) + public ApplicationConfigNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, string baseFileName, IDictionary environmentVariables, IDictionary systemProperties) + : base (targetProvider, baseFileName) { if (environmentVariables != null) this.environmentVariables = new SortedDictionary (environmentVariables, StringComparer.Ordinal); @@ -44,7 +45,7 @@ protected override void WriteSymbols (StreamWriter output) WriteData (output, AndroidPackageName, stringLabel); WriteDataSection (output, "application_config"); - WriteSymbol (output, "application_config", TargetProvider.GetStructureAlignment (true), fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { + WriteSymbol (output, "application_config", TargetProvider.GetStructureAlignment (true), packed: false, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { // Order of fields and their type must correspond *exactly* to that in // src/monodroid/jni/xamarin-app.h ApplicationConfig structure WriteCommentLine (output, "uses_mono_llvm"); @@ -62,6 +63,9 @@ protected override void WriteSymbols (StreamWriter output) WriteCommentLine (output, "broken_exception_transitions"); size += WriteData (output, BrokenExceptionTransitions); + WriteCommentLine (output, "instant_run_enabled"); + size += WriteData (output, InstantRunEnabled); + WriteCommentLine (output, "bound_exception_type"); size += WriteData (output, (byte)BoundExceptionType); @@ -74,13 +78,8 @@ protected override void WriteSymbols (StreamWriter output) WriteCommentLine (output, "system_property_count"); size += WriteData (output, systemProperties == null ? 0 : systemProperties.Count * 2); - // After uses_embedded_dsos was removed, we need padding on 64-bit - if (TargetProvider.Is64Bit) { - size += WriteDataPadding (output, 4); - } - WriteCommentLine (output, "android_package_name"); - size += WritePointer (output, stringLabel); + size += WritePointer (output, MakeLocalLabel (stringLabel)); return size; }); @@ -88,7 +87,7 @@ protected override void WriteSymbols (StreamWriter output) stringLabel = GetStringLabel (); WriteData (output, MonoAOTMode ?? String.Empty, stringLabel); WriteDataSection (output, "mono_aot_mode_name"); - WritePointer (output, stringLabel, "mono_aot_mode_name", isGlobal: true); + WritePointer (output, MakeLocalLabel (stringLabel), "mono_aot_mode_name", isGlobal: true); WriteNameValueStringArray (output, "app_environment_variables", environmentVariables); WriteNameValueStringArray (output, "app_system_properties", systemProperties); @@ -98,7 +97,7 @@ void WriteNameValueStringArray (StreamWriter output, string label, SortedDiction { if (entries == null || entries.Count == 0) { WriteDataSection (output, label); - WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: null); + WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), packed: false, isGlobal: true, alwaysWriteSize: true, structureWriter: null); return; } @@ -117,11 +116,11 @@ void WriteNameValueStringArray (StreamWriter output, string label, SortedDiction } WriteDataSection (output, label); - WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { + WriteSymbol (output, label, TargetProvider.GetStructureAlignment (true), packed: false, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { uint size = 0; foreach (string l in entry_labels) { - size += WritePointer (output, l); + size += WritePointer (output, MakeLocalLabel (l)); } return size; @@ -136,7 +135,7 @@ void WriteDataSection (StreamWriter output, string tag) string GetStringLabel () { stringCounter++; - return $".L.str.{stringCounter}"; + return $"env.str.{stringCounter}"; } }; } diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs b/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs index a5f778d5af4..0cc1af3b8d8 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs @@ -307,7 +307,7 @@ public static bool IsFrameworkAssembly (string assembly, bool checkSdkPath) { if (IsSharedRuntimeAssembly (assembly)) { #if MSBUILD - bool treatAsUser = Array.BinarySearch (FrameworkAssembliesToTreatAsUserAssemblies, Path.GetFileName (assembly), StringComparer.OrdinalIgnoreCase) >= 0; + bool treatAsUser = FrameworkAssembliesToTreatAsUserAssemblies.Contains (Path.GetFileName (assembly)); // Framework assemblies don't come from outside the SDK Path; // user assemblies do if (checkSdkPath && treatAsUser && TargetFrameworkDirectories != null) { @@ -527,10 +527,14 @@ public static bool IsRawResourcePath (string projectPath) #if MSBUILD internal static IEnumerable GetFrameworkAssembliesToTreatAsUserAssemblies (ITaskItem[] resolvedAssemblies) - { - return resolvedAssemblies - .Where (f => Array.BinarySearch (FrameworkAssembliesToTreatAsUserAssemblies, Path.GetFileName (f.ItemSpec), StringComparer.OrdinalIgnoreCase) >= 0) - .Select(p => p); + { + var ret = new List (); + foreach (ITaskItem item in resolvedAssemblies) { + if (FrameworkAssembliesToTreatAsUserAssemblies.Contains (Path.GetFileName (item.ItemSpec))) + ret.Add (item); + } + + return ret; } #endif @@ -543,8 +547,7 @@ internal static IEnumerable GetFrameworkAssembliesToTreatAsUserAssemb "Mono.Data.Sqlite.dll", "Mono.Posix.dll", }; - // MUST BE SORTED CASE-INSENSITIVE - internal static readonly string[] FrameworkAssembliesToTreatAsUserAssemblies = { + internal static readonly HashSet FrameworkAssembliesToTreatAsUserAssemblies = new HashSet (StringComparer.OrdinalIgnoreCase) { "Mono.Android.Support.v13.dll", "Mono.Android.Support.v4.dll", "Xamarin.Android.NUnitLite.dll", diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs index d3158abf8fc..6b379232078 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblerTargetProvider.cs @@ -8,6 +8,9 @@ abstract class NativeAssemblerTargetProvider public abstract bool Is64Bit { get; } public abstract string PointerFieldType { get; } public abstract string TypePrefix { get; } + public abstract string AbiName { get; } + public abstract uint MapModulesAlignBits { get; } + public abstract uint MapJavaAlignBits { get; } public virtual string MapType () { @@ -30,22 +33,27 @@ abstract class NativeAssemblerTargetProvider public virtual uint GetTypeSize () { - if (typeof(T) == typeof(byte)) + return GetTypeSize (typeof(T)); + } + + public virtual uint GetTypeSize (Type type) + { + if (type == typeof(byte)) return 1u; - if (typeof(T) == typeof(bool)) + if (type == typeof(bool)) return 1u; - if (typeof(T) == typeof(string)) + if (type == typeof(string)) return GetPointerSize(); - if (typeof(T) == typeof(Int32) || typeof(T) == typeof(UInt32)) + if (type == typeof(Int32) || type == typeof(UInt32)) return 4u; - if (typeof(T) == typeof(Int64) || typeof(T) == typeof(UInt64)) + if (type == typeof(Int64) || type == typeof(UInt64)) return 8u; - throw new InvalidOperationException ($"Unable to map managed type {typeof(T)} to native assembly type"); + throw new InvalidOperationException ($"Unable to map managed type {type} to native assembly type"); } public virtual uint GetStructureAlignment (bool hasPointers) diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs deleted file mode 100644 index f0e3e6526a4..00000000000 --- a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyDataStream.cs +++ /dev/null @@ -1,235 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Security.Cryptography; -using System.IO; -using System.Text; -using Java.Interop.Tools.JavaCallableWrappers; - -namespace Xamarin.Android.Tasks -{ - class NativeAssemblyDataStream : Stream - { - const uint MapVersionFound = 0x0001; - const uint MapEntryCountFound = 0x0002; - const uint MapEntryLengthFound = 0x0004; - const uint MapValueOffsetFound = 0x0008; - const uint MapEverythingFound = MapVersionFound | MapEntryCountFound | MapEntryLengthFound | MapValueOffsetFound; - - const string MapFieldVersion = "version"; - const string MapFieldEntryCount = "entry-count"; - const string MapFieldEntryLength = "entry-len"; - const string MapFieldValueOffset = "value-offset"; - - const uint MaxFieldsInRow = 32; - - MemoryStream outputStream; - StreamWriter outputWriter; - bool firstWrite = true; - uint rowFieldCounter; - HashAlgorithm hashAlgorithm; - bool hashingFinished; - uint byteCount = 0; - - public int MapVersion { get; set; } = -1; - public int MapEntryCount { get; set; } = -1; - public int MapEntryLength { get; set; } = -1; - public int MapValueOffset { get; set; } = -1; - public uint MapByteCount => byteCount; - - public override bool CanRead => outputStream.CanRead; - public override bool CanSeek => outputStream.CanSeek; - public override bool CanWrite => outputStream.CanWrite; - public override long Length => outputStream.Length; - - public override long Position { - get => outputStream.Position; - set => outputStream.Position = value; - } - - public NativeAssemblyDataStream () - { - outputStream = new MemoryStream (); - outputWriter = new StreamWriter (outputStream, new UTF8Encoding (false)); - hashAlgorithm = new Crc64 (); - hashAlgorithm.Initialize (); - } - - public byte[] GetStreamHash () - { - if (!hashingFinished) { - hashAlgorithm.TransformFinalBlock (new byte[0], 0, 0); - hashingFinished = true; - } - - return hashAlgorithm.Hash; - } - - protected override void Dispose (bool disposing) - { - base.Dispose (disposing); - if (!disposing) - return; - - outputWriter.Dispose (); - outputWriter = null; - outputStream = null; - } - - public void EndOfFile () - { - outputWriter.WriteLine (); - Flush (); - } - - public void EmptyFile () - { - MapVersion = 1; - MapEntryCount = 0; - MapEntryLength = 0; - MapValueOffset = 0; - } - - public override void Flush () - { - outputWriter.Flush (); - outputStream.Flush (); - } - - public override int Read (byte[] buffer, int offset, int count) - { - return outputStream.Read (buffer, offset, count); - } - - public override long Seek (long offset, SeekOrigin origin) - { - return outputStream.Seek (offset, origin); - } - - public override void SetLength (long value) - { - outputStream.SetLength (value); - } - - public override void Write (byte[] buffer, int offset, int count) - { - if (buffer == null) - throw new ArgumentNullException (nameof (buffer)); - if (offset < 0) - throw new ArgumentOutOfRangeException (nameof (offset)); - if (count < 0) - throw new ArgumentOutOfRangeException (nameof (count)); - if (offset + count > buffer.Length) - throw new ArgumentException ($"The sum of the '{nameof (offset)}' and '{nameof (count)}' arguments is greater than the buffer length"); - if (outputWriter == null) - throw new ObjectDisposedException (this.GetType ().Name); - - if (!hashingFinished) - hashAlgorithm.TransformBlock (buffer, offset, count, null, 0); - - if (firstWrite) { - // Kind of a backward thing to do, but I wanted to avoid having to modfy (and bump the - // submodule of) Java.Interop. This should be Safe Enough™ (until it's not) - string line = Encoding.UTF8.GetString (buffer, offset, count); - - // Format used by Java.Interop.Tools.JavaCallableWrappers.TypeNameMapGenerator: - // - // "version=1\u0000entry-count={0}\u0000entry-len={1}\u0000value-offset={2}\u0000" - // - string[] parts = line.Split ('\u0000'); - if (parts.Length != 5) - HeaderFormatError ("invalid number of fields"); - - // Let's be just slightly flexible :P - uint foundMask = 0; - foreach (string p in parts) { - string field = p.Trim (); - if (String.IsNullOrEmpty (field)) - continue; - - string[] fieldParts = field.Split(new char[]{'='}, 2); - if (fieldParts.Length != 2) - HeaderFormatError ($"invalid field format '{field}'"); - - switch (fieldParts [0]) { - case MapFieldVersion: - foundMask |= MapVersionFound; - MapVersion = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - case MapFieldEntryCount: - foundMask |= MapEntryCountFound; - MapEntryCount = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - case MapFieldEntryLength: - foundMask |= MapEntryLengthFound; - MapEntryLength = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - case MapFieldValueOffset: - foundMask |= MapValueOffsetFound; - MapValueOffset = GetHeaderInteger (fieldParts[0], fieldParts [1]); - break; - - default: - HeaderFormatError ($"unknown field '{fieldParts [0]}'"); - break; - } - } - - if (foundMask != MapEverythingFound) { - var missingFields = new List (); - if ((foundMask & MapVersionFound) == 0) - missingFields.Add (MapFieldVersion); - if ((foundMask & MapEntryCountFound) == 0) - missingFields.Add (MapFieldEntryCount); - if ((foundMask & MapEntryLengthFound) == 0) - missingFields.Add (MapFieldEntryLength); - if ((foundMask & MapValueOffsetFound) == 0) - missingFields.Add (MapFieldValueOffset); - - var sb = new StringBuilder ("missing header field"); - if (missingFields.Count > 1) - sb.Append ('s'); - sb.Append (": "); - sb.Append (String.Join (", ", missingFields)); - HeaderFormatError (sb.ToString ()); - } - - firstWrite = false; - rowFieldCounter = 0; - return; - } - - for (int i = 0; i < count; i++) { - byteCount++; - if (rowFieldCounter == 0) - outputWriter.Write ("\t.byte "); - - byte b = buffer [offset + i]; - if (rowFieldCounter > 0) - outputWriter.Write (", "); - outputWriter.Write ($"0x{b:x02}"); - - rowFieldCounter++; - if (rowFieldCounter > MaxFieldsInRow) { - rowFieldCounter = 0; - outputWriter.WriteLine (); - } - } - - void HeaderFormatError (string whatsWrong) - { - throw new InvalidOperationException ($"Java.Interop.Tools.JavaCallableWrappers.TypeNameMapGenerator header format changed: {whatsWrong}"); - } - - int GetHeaderInteger (string name, string value) - { - int ret; - if (!Int32.TryParse (value, out ret)) - HeaderFormatError ($"failed to parse integer value from '{value}' for field '{name}'"); - return ret; - } - } - }; -} diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs index 17bfca35db7..3b8b8a54d5c 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/NativeAssemblyGenerator.cs @@ -7,39 +7,49 @@ namespace Xamarin.Android.Tasks abstract class NativeAssemblyGenerator { uint structureByteCount; - uint structureAlignBytes; + bool structureIsPacked = false; bool writingStructure = false; protected string Indent { get; } = "\t"; protected NativeAssemblerTargetProvider TargetProvider { get; } + protected string TypemapsIncludeFile { get; } + protected string SharedIncludeFile { get; } + public string MainSourceFile { get; } - protected NativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider) + protected NativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, string baseFilePath, bool sharedIncludeUsesAbiPrefix = false) { if (targetProvider == null) throw new ArgumentNullException (nameof (targetProvider)); + if (String.IsNullOrEmpty (baseFilePath)) + throw new ArgumentException ("must not be null or empty", nameof (baseFilePath)); + TargetProvider = targetProvider; + if (sharedIncludeUsesAbiPrefix) + SharedIncludeFile = $"{baseFilePath}.{targetProvider.AbiName}-shared.inc"; + else + SharedIncludeFile = $"{baseFilePath}.shared.inc"; + TypemapsIncludeFile = $"{baseFilePath}.{targetProvider.AbiName}-managed.inc"; + MainSourceFile = $"{baseFilePath}.{targetProvider.AbiName}.s"; } - public void Write (StreamWriter output, string outputFileName) + public void Write (StreamWriter output) { if (output == null) throw new ArgumentNullException (nameof (output)); - WriteFileHeader (output, outputFileName); + WriteFileHeader (output); WriteSymbols (output); + WriteFileFooter (output); output.Flush (); } protected virtual void WriteSymbols (StreamWriter output) - { - } + {} protected void WriteEndLine (StreamWriter output, string comment = null, bool indent = true) { if (!String.IsNullOrEmpty (comment)) { - if (indent) - output.Write (Indent); - WriteComment (output, comment); + WriteComment (output, comment, indent); } output.WriteLine (); } @@ -51,16 +61,19 @@ protected void WriteComment (StreamWriter output, string comment, bool indent = protected void WriteCommentLine (StreamWriter output, string comment, bool indent = true) { - WriteComment (output, comment); + WriteComment (output, comment, indent); output.WriteLine (); } - protected virtual void WriteFileHeader (StreamWriter output, string outputFileName) + protected virtual void WriteFileHeader (StreamWriter output) { TargetProvider.WriteFileHeader (output, Indent); - output.WriteLine ($"{Indent}.file{Indent}\"{Path.GetFileName (outputFileName)}\""); + output.WriteLine ($"{Indent}.file{Indent}\"{Path.GetFileName (MainSourceFile)}\""); } + protected virtual void WriteFileFooter (StreamWriter output) + {} + protected virtual void WriteSection (StreamWriter output, string sectionName, bool hasStrings, bool writable) { output.Write ($"{Indent}.section{Indent}{sectionName},\"a"); // a - section is allocatable @@ -106,7 +119,19 @@ protected void WriteSymbol (StreamWriter output, string label, uint size, bool i WriteSymbol (output, null, null, label, size, alignBits: 0, isGlobal: isGlobal, isObject: isObject, alwaysWriteSize: alwaysWriteSize); } - protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBits, uint fieldAlignBytes, bool isGlobal, bool alwaysWriteSize, Func structureWriter) + protected void WriteSymbol (StreamWriter output, string label, uint size, uint alignBits, bool isGlobal, bool isObject, bool alwaysWriteSize) + { + WriteSymbol (output, null, null, label, size, alignBits, isGlobal: isGlobal, isObject: isObject, alwaysWriteSize: alwaysWriteSize); + } + + protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBits, bool packed, bool isGlobal, bool alwaysWriteSize, Func structureWriter) + { + WriteStructureSymbol (output, symbolName, alignBits, isGlobal); + uint size = WriteStructure (output, packed, structureWriter); + WriteStructureSize (output, symbolName, size, alwaysWriteSize); + } + + protected void WriteStructureSymbol (StreamWriter output, string symbolName, uint alignBits, bool isGlobal) { output.WriteLine ($"{Indent}.type{Indent}{symbolName}, {TargetProvider.TypePrefix}object"); if (alignBits > 0) @@ -115,14 +140,28 @@ protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBi output.WriteLine ($"{Indent}.global{Indent}{symbolName}"); if (!String.IsNullOrEmpty (symbolName)) output.WriteLine ($"{symbolName}:"); + } + + protected void WriteStructureSize (StreamWriter output, string symbolName, uint size, bool alwaysWriteSize = false) + { + if (size == 0 && !alwaysWriteSize) + return; + if (String.IsNullOrEmpty (symbolName)) + throw new ArgumentException ("symbol name must be non-empty in order to write structure size", nameof (symbolName)); + + output.WriteLine ($"{Indent}.size{Indent}{symbolName}, {size}"); + } + + protected uint WriteStructure (StreamWriter output, bool packed, Func structureWriter) + { writingStructure = true; structureByteCount = 0; - structureAlignBytes = fieldAlignBytes; + structureIsPacked = packed; uint size = structureWriter != null ? structureWriter () : 0u; writingStructure = false; - if (alwaysWriteSize || size > 0) - output.WriteLine ($"{Indent}.size{Indent}{symbolName}, {size}"); + + return size; } protected virtual string QuoteValue (T value) @@ -136,31 +175,38 @@ protected void WriteSymbol (StreamWriter output, string symbolName, uint alignBi return $"{value}"; } - protected uint WritePointer (StreamWriter output, string targetName, string label = null, bool isGlobal = false) + protected uint WritePointer (StreamWriter output, string targetName = null, string label = null, bool isGlobal = false) { - if (String.IsNullOrEmpty (targetName)) - throw new ArgumentException ("must not be null or empty", nameof (targetName)); - - uint fieldSize = UpdateSize (output, targetName); - WriteSymbol (output, TargetProvider.PointerFieldType, targetName, label, size: 0, alignBits: 0, isGlobal: isGlobal, isObject: false, alwaysWriteSize: false); + uint fieldSize = UpdateSize (output, targetName ?? String.Empty); + WriteSymbol (output, TargetProvider.PointerFieldType, targetName ?? "0", label, size: 0, alignBits: 0, isGlobal: isGlobal, isObject: false, alwaysWriteSize: false); return fieldSize; } uint WriteDataPadding (StreamWriter output, uint nextFieldSize, uint sizeSoFar) { - if (!writingStructure || structureAlignBytes <= 1 || nextFieldSize == 1) + if (!writingStructure || structureIsPacked || nextFieldSize == 1) return 0; - uint alignment = sizeSoFar % structureAlignBytes; + uint modulo; + if (TargetProvider.Is64Bit) { + modulo = nextFieldSize < 8 ? 4u : 8u; + } else { + modulo = 4u; + } + + uint alignment = sizeSoFar % modulo; if (alignment == 0) return 0; - uint nbytes = structureAlignBytes - alignment; + uint nbytes = modulo - alignment; return WriteDataPadding (output, nbytes); } protected uint WriteDataPadding (StreamWriter output, uint nbytes) { + if (nbytes == 0) + return 0; + output.WriteLine ($"{Indent}.zero{Indent}{nbytes}"); return nbytes; } @@ -173,24 +219,78 @@ protected uint WriteData (StreamWriter output, string value, string label, bool value = String.Empty; WriteSection (output, $".rodata.{label}", hasStrings: true, writable: false); - WriteSymbol (output, value, label, size: (ulong)(value.Length + 1), alignBits: 0, isGlobal: isGlobal, isObject: true, alwaysWriteSize: true); + WriteSymbol (output, value, isGlobal ? label : MakeLocalLabel (label), size: (ulong)(value.Length + 1), alignBits: 0, isGlobal: isGlobal, isObject: true, alwaysWriteSize: true); return TargetProvider.GetTypeSize (value); } + protected uint WriteAsciiData (StreamWriter output, string value, uint padToWidth = 0) + { + if (value == null) + value = String.Empty; + + uint size = (uint)output.Encoding.GetByteCount (value); + if (size > 0) { + output.WriteLine ($"{Indent}.ascii{Indent}\"{value}\""); + } else if (padToWidth == 0) { + return WriteDataPadding (output, 1); + } + + if (padToWidth > size) + size += WriteDataPadding (output, padToWidth - size); + + return size; + } + + protected string MakeLocalLabel (string label) + { + return $".L.{label}"; + } + + uint UpdateSize (StreamWriter output, T[] value) + { + uint typeSize = TargetProvider.GetTypeSize (); + uint fieldSize = typeSize * (uint)value.Length; + + fieldSize += WriteDataPadding (output, fieldSize, structureByteCount); + structureByteCount += fieldSize; + + return fieldSize; + } + uint UpdateSize (StreamWriter output, T value) { uint fieldSize; - if (typeof (T) == typeof (string)) + Type t = typeof(T); + if (t == typeof (string)) fieldSize = TargetProvider.GetPointerSize (); else fieldSize = TargetProvider.GetTypeSize (value); + fieldSize += WriteDataPadding (output, fieldSize, structureByteCount); structureByteCount += fieldSize; return fieldSize; } + protected uint WriteData (StreamWriter output, byte[] value, string label = null, bool isGlobal = false) + { + uint fieldSize = UpdateSize (output, value); + var symbolValue = new StringBuilder (); + bool first = true; + foreach (byte b in value) { + if (!first) + symbolValue.Append (", "); + else + first = false; + + symbolValue.Append ($"0x{b:x02}"); + } + WriteSymbol (output, TargetProvider.MapType (), symbolValue.ToString (), symbolName: label, size: 0, alignBits: 0, isGlobal: isGlobal, isObject: false, alwaysWriteSize: false); + + return fieldSize; + } + protected uint WriteData (StreamWriter output, byte value, string label = null, bool isGlobal = false) { uint fieldSize = UpdateSize (output, value); diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs b/src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs new file mode 100644 index 00000000000..c915d6d46a8 --- /dev/null +++ b/src/Xamarin.Android.Build.Tasks/Utilities/NativeTypeMappingData.cs @@ -0,0 +1,60 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace Xamarin.Android.Tasks +{ + class NativeTypeMappingData + { + public TypeMapGenerator.ModuleData[] Modules { get; } + public IDictionary AssemblyNames { get; } + public string[] JavaTypeNames { get; } + public TypeMapGenerator.TypeMapEntry[] JavaTypes { get; } + + public uint MapModuleCount { get; } + public uint JavaTypeCount { get; } + public uint JavaNameWidth { get; } + + public NativeTypeMappingData (Action logger, TypeMapGenerator.ModuleData[] modules, int javaNameWidth) + { + Modules = modules ?? throw new ArgumentNullException (nameof (modules)); + + MapModuleCount = (uint)modules.Length; + JavaNameWidth = (uint)javaNameWidth; + + AssemblyNames = new Dictionary (StringComparer.Ordinal); + + var tempJavaTypes = new Dictionary (StringComparer.Ordinal); + int managedStringCounter = 0; + var moduleComparer = new TypeMapGenerator.ModuleUUIDArrayComparer (); + + foreach (TypeMapGenerator.ModuleData data in modules) { + data.AssemblyNameLabel = $"map_aname.{managedStringCounter++}"; + AssemblyNames.Add (data.AssemblyNameLabel, data.AssemblyName); + + int moduleIndex = Array.BinarySearch (modules, data, moduleComparer); + if (moduleIndex < 0) + throw new InvalidOperationException ($"Unable to map module with MVID {data.Mvid} to array index"); + + foreach (TypeMapGenerator.TypeMapEntry entry in data.Types) { + entry.ModuleIndex = moduleIndex; + if (tempJavaTypes.ContainsKey (entry.JavaName)) + continue; + tempJavaTypes.Add (entry.JavaName, entry); + } + } + + var javaNames = tempJavaTypes.Keys.ToArray (); + Array.Sort (javaNames, StringComparer.Ordinal); + + var javaTypes = new TypeMapGenerator.TypeMapEntry[javaNames.Length]; + for (int i = 0; i < javaNames.Length; i++) { + javaTypes[i] = tempJavaTypes[javaNames[i]]; + } + + JavaTypes = javaTypes; + JavaTypeNames = javaNames; + JavaTypeCount = (uint)JavaTypes.Length; + } + } +} diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs new file mode 100644 index 00000000000..f397f57019c --- /dev/null +++ b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs @@ -0,0 +1,423 @@ +using System; +using System.IO; +using System.Collections.Generic; +using System.Linq; +using System.Text; + +using Mono.Cecil; + +namespace Xamarin.Android.Tasks +{ + class TypeMapGenerator + { + const string TypeMapMagicString = "XATM"; // Xamarin Android TypeMap + const string TypeMapIndexMagicString = "XATI"; // Xamarin Android Typemap Index + const uint TypeMapFormatVersion = 1; // Keep in sync with the value in src/monodroid/jni/xamarin-app.hh + + internal sealed class ModuleUUIDArrayComparer : IComparer + { + int Compare (byte[] left, byte[] right) + { + int ret; + + for (int i = 0; i < 16; i++) { + ret = left[i].CompareTo (right[i]); + if (ret != 0) + return ret; + } + + return 0; + } + + public int Compare (ModuleData left, ModuleData right) + { + return Compare (left.MvidBytes, right.MvidBytes); + } + } + + internal sealed class TypeMapEntryArrayComparer : IComparer + { + public int Compare (TypeMapEntry left, TypeMapEntry right) + { + return String.CompareOrdinal (left.JavaName, right.JavaName); + } + } + + internal sealed class TypeMapEntry + { + public string JavaName; + public int JavaNameLength; + public string ManagedTypeName; + public uint Token; + public int AssemblyNameIndex = -1; + public int ModuleIndex = -1; + } + + internal sealed class ModuleData + { + public Guid Mvid; + public byte[] MvidBytes; + public AssemblyDefinition Assembly; + public TypeMapEntry[] Types; + public Dictionary DuplicateTypes; + public string AssemblyName; + public string AssemblyNameLabel; + public string OutputFilePath; + + public Dictionary TypesScratch; + } + + Action logger; + Encoding outputEncoding; + byte[] moduleMagicString; + byte[] typemapIndexMagicString; + string[] supportedAbis; + + public IList GeneratedBinaryTypeMaps { get; } = new List (); + + public TypeMapGenerator (Action logger, string[] supportedAbis) + { + this.logger = logger ?? throw new ArgumentNullException (nameof (logger)); + if (supportedAbis == null) + throw new ArgumentNullException (nameof (supportedAbis)); + this.supportedAbis = supportedAbis; + + outputEncoding = new UTF8Encoding (false); + moduleMagicString = outputEncoding.GetBytes (TypeMapMagicString); + typemapIndexMagicString = outputEncoding.GetBytes (TypeMapIndexMagicString); + } + + public bool Generate (List javaTypes, string outputDirectory, bool generateNativeAssembly) + { + if (String.IsNullOrEmpty (outputDirectory)) + throw new ArgumentException ("must not be null or empty", nameof (outputDirectory)); + + if (!Directory.Exists (outputDirectory)) + Directory.CreateDirectory (outputDirectory); + + int assemblyId = 0; + int maxJavaNameLength = 0; + int maxModuleFileNameLength = 0; + var knownAssemblies = new Dictionary (StringComparer.Ordinal); + var tempModules = new Dictionary (); + Dictionary moduleCounter = null; + var mvidCache = new Dictionary (); + + foreach (TypeDefinition td in javaTypes) { + string assemblyName = td.Module.Assembly.FullName; + + if (!knownAssemblies.ContainsKey (assemblyName)) { + assemblyId++; + knownAssemblies.Add (assemblyName, assemblyId); + } + + // We must NOT use Guid here! The reason is that Guid sort order is different than its corresponding + // byte array representation and on the runtime we need the latter in order to be able to binary search + // through the module array. + byte[] moduleUUID; + if (!mvidCache.TryGetValue (td.Module.Mvid, out moduleUUID)) { + moduleUUID = td.Module.Mvid.ToByteArray (); + mvidCache.Add (td.Module.Mvid, moduleUUID); + } + + ModuleData moduleData; + if (!tempModules.TryGetValue (moduleUUID, out moduleData)) { + if (moduleCounter == null) + moduleCounter = new Dictionary (); + + moduleData = new ModuleData { + Mvid = td.Module.Mvid, + MvidBytes = moduleUUID, + Assembly = td.Module.Assembly, + AssemblyName = td.Module.Assembly.Name.Name, + TypesScratch = new Dictionary (StringComparer.Ordinal), + DuplicateTypes = new Dictionary (), + }; + tempModules.Add (moduleUUID, moduleData); + + if (!generateNativeAssembly) { + int moduleNum; + if (!moduleCounter.TryGetValue (moduleData.Assembly, out moduleNum)) { + moduleNum = 0; + moduleCounter [moduleData.Assembly] = 0; + } else { + moduleNum++; + moduleCounter [moduleData.Assembly] = moduleNum; + } + + string fileName = $"{moduleData.Assembly.Name.Name}.{moduleNum}.typemap"; + moduleData.OutputFilePath = Path.Combine (outputDirectory, fileName); + if (maxModuleFileNameLength < fileName.Length) + maxModuleFileNameLength = fileName.Length; + } + } + + string javaName = Java.Interop.Tools.TypeNameMappings.JavaNativeTypeManager.ToJniName (td); + var entry = new TypeMapEntry { + JavaName = javaName, + JavaNameLength = outputEncoding.GetByteCount (javaName), + ManagedTypeName = td.FullName, + Token = td.MetadataToken.ToUInt32 (), + AssemblyNameIndex = knownAssemblies [assemblyName] + }; + + if (generateNativeAssembly) { + if (entry.JavaNameLength > maxJavaNameLength) + maxJavaNameLength = entry.JavaNameLength; + } + + if (moduleData.TypesScratch.ContainsKey (entry.JavaName)) { + // This is disabled because it costs a lot of time (around 150ms per standard XF Integration app + // build) and has no value for the end user. The message is left here because it may be useful to us + // in our devloop at some point. + //logger ($"Warning: duplicate Java type name '{entry.JavaName}' in assembly '{moduleData.AssemblyName}' (new token: {entry.Token})."); + moduleData.DuplicateTypes.Add (entry.Token, entry); + } else + moduleData.TypesScratch.Add (entry.JavaName, entry); + } + + var modules = tempModules.Values.ToArray (); + Array.Sort (modules, new ModuleUUIDArrayComparer ()); + + var typeMapEntryComparer = new TypeMapEntryArrayComparer (); + foreach (ModuleData module in modules) { + if (module.TypesScratch.Count == 0) { + module.Types = new TypeMapEntry[0]; + continue; + } + + module.Types = module.TypesScratch.Values.ToArray (); + Array.Sort (module.Types, typeMapEntryComparer); + } + + NativeTypeMappingData data; + if (!generateNativeAssembly) { + string typeMapIndexPath = Path.Combine (outputDirectory, "typemap.index"); + // Try to approximate the index size: + // 16 bytes for the header + // 16 bytes (UUID) + filename length per each entry + using (var ms = new MemoryStream (16 + (modules.Length * (16 + 128)))) { + using (var indexWriter = new BinaryWriter (ms)) { + OutputModules (outputDirectory, modules, indexWriter, maxModuleFileNameLength + 1); + indexWriter.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, typeMapIndexPath); + } + } + GeneratedBinaryTypeMaps.Add (typeMapIndexPath); + + data = new NativeTypeMappingData (logger, new ModuleData[0], 0); + } else { + data = new NativeTypeMappingData (logger, modules, maxJavaNameLength + 1); + } + + NativeAssemblerTargetProvider asmTargetProvider; + bool sharedBitsWritten = false; + bool sharedIncludeUsesAbiPrefix; + foreach (string abi in supportedAbis) { + sharedIncludeUsesAbiPrefix = false; + switch (abi.Trim ()) { + case "armeabi-v7a": + asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: false); + sharedIncludeUsesAbiPrefix = true; // ARMv7a is "special", it uses different directive prefix + // than the others and the "shared" code won't build for it + break; + + case "arm64-v8a": + asmTargetProvider = new ARMNativeAssemblerTargetProvider (is64Bit: true); + break; + + case "x86": + asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: false); + break; + + case "x86_64": + asmTargetProvider = new X86NativeAssemblerTargetProvider (is64Bit: true); + break; + + default: + throw new InvalidOperationException ($"Unknown ABI {abi}"); + } + + var generator = new TypeMappingNativeAssemblyGenerator (asmTargetProvider, data, Path.Combine (outputDirectory, "typemaps"), sharedBitsWritten, sharedIncludeUsesAbiPrefix); + + using (var ms = new MemoryStream ()) { + using (var sw = new StreamWriter (ms, outputEncoding)) { + generator.Write (sw); + sw.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, generator.MainSourceFile); + if (!sharedIncludeUsesAbiPrefix) + sharedBitsWritten = true; + } + } + } + return true; + } + + // Binary index file format, all data is little-endian: + // + // [Magic string] # XATI + // [Format version] # 32-bit unsigned integer, 4 bytes + // [Entry count] # 32-bit unsigned integer, 4 bytes + // [Module file name width] # 32-bit unsigned integer, 4 bytes + // [Index entries] # Format described below, [Entry count] entries + // + // Index entry format: + // + // [Module UUID][File name] + // + // Where: + // + // [Module UUID] is 16 bytes long + // [File name] is right-padded with characters to the [Module file name width] boundary. + // + void OutputModules (string outputDirectory, ModuleData[] modules, BinaryWriter indexWriter, int moduleFileNameWidth) + { + var moduleCounter = new Dictionary (); + + indexWriter.Write (typemapIndexMagicString); + indexWriter.Write (TypeMapFormatVersion); + indexWriter.Write (modules.Length); + indexWriter.Write (moduleFileNameWidth); + + foreach (ModuleData data in modules) { + OutputModule (outputDirectory, data.MvidBytes, data, moduleCounter); + indexWriter.Write (data.MvidBytes); + + string outputFilePath = Path.GetFileName (data.OutputFilePath); + indexWriter.Write (outputEncoding.GetBytes (outputFilePath)); + PadField (indexWriter, outputFilePath.Length, moduleFileNameWidth); + } + } + + void OutputModule (string outputDirectory, byte[] moduleUUID, ModuleData moduleData, Dictionary moduleCounter) + { + if (moduleData.Types.Length == 0) + return; + + int initialStreamSize = + (36 + 128) + // Static header size + assembly file name + ((128 + 4) * moduleData.Types.Length) + // java-to-managed size + (8 * moduleData.Types.Length) + // managed-to-java size + (8 * moduleData.DuplicateTypes.Count); // managed-to-java duplicates; + + using (var ms = new MemoryStream (initialStreamSize)) { + using (var bw = new BinaryWriter (ms)) { + OutputModule (bw, moduleUUID, moduleData); + bw.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, moduleData.OutputFilePath); + } + } + GeneratedBinaryTypeMaps.Add (moduleData.OutputFilePath); + } + + // Binary file format, all data is little-endian: + // + // [Magic string] # XATM + // [Format version] # 32-bit integer, 4 bytes + // [Module UUID] # 16 bytes + // [Entry count] # unsigned 32-bit integer, 4 bytes + // [Duplicate count] # unsigned 32-bit integer, 4 bytes (might be 0) + // [Java type name width] # unsigned 32-bit integer, 4 bytes + // [Assembly name size] # unsigned 32-bit integer, 4 bytes + // [Assembly name] # Non-null terminated assembly name + // [Java-to-managed map] # Format described below, [Entry count] entries + // [Managed-to-java map] # Format described below, [Entry count] entries + // [Managed-to-java duplicates map] # Map of unique managed IDs which point to the same Java type name (might be empty) + // + // Java-to-managed map format: + // + // [Java type name][Managed type token ID] + // + // Each name is padded with to the width specified in the [Java type name width] field above. + // Names are written without the size prefix, instead they are always terminated with a nul character + // to make it easier and faster to handle by the native runtime. + // + // Each token ID is an unsigned 32-bit integer, 4 bytes + // + // + // Managed-to-java map format: + // + // [Managed type token ID][Java type name table index] + // + // Both fields are unsigned 32-bit integers, to a total of 8 bytes per entry. Index points into the + // [Java-to-managed map] table above. + // + // Managed-to-java duplicates map format: + // + // Format is identical to [Managed-to-java] above. + // + void OutputModule (BinaryWriter bw, byte[] moduleUUID, ModuleData moduleData) + { + bw.Write (moduleMagicString); + bw.Write (TypeMapFormatVersion); + bw.Write (moduleUUID); + + var javaNames = new Dictionary (StringComparer.Ordinal); + var managedTypes = new Dictionary (); + int maxJavaNameLength = 0; + + foreach (TypeMapEntry entry in moduleData.Types) { + javaNames.Add (entry.JavaName, entry.Token); + if (entry.JavaNameLength > maxJavaNameLength) + maxJavaNameLength = entry.JavaNameLength; + + managedTypes.Add (entry.Token, 0); + } + + var javaNameList = javaNames.Keys.ToList (); + foreach (TypeMapEntry entry in moduleData.Types) { + var javaIndex = (uint)javaNameList.IndexOf (entry.JavaName); + managedTypes[entry.Token] = javaIndex; + } + + bw.Write (javaNames.Count); + bw.Write (moduleData.DuplicateTypes.Count); + bw.Write (maxJavaNameLength + 1); + + string assemblyName = moduleData.Assembly.Name.Name; + bw.Write (assemblyName.Length); + bw.Write (outputEncoding.GetBytes (assemblyName)); + + var sortedJavaNames = javaNames.Keys.ToArray (); + Array.Sort (sortedJavaNames, StringComparer.Ordinal); + foreach (string typeName in sortedJavaNames) { + byte[] bytes = outputEncoding.GetBytes (typeName); + bw.Write (bytes); + PadField (bw, bytes.Length, maxJavaNameLength + 1); + bw.Write (javaNames[typeName]); + } + + WriteManagedTypes (managedTypes); + if (moduleData.DuplicateTypes.Count == 0) + return; + + var managedDuplicates = new Dictionary (); + foreach (var kvp in moduleData.DuplicateTypes) { + uint javaIndex = kvp.Key; + uint typeId = kvp.Value.Token; + + managedDuplicates.Add (javaIndex, typeId); + } + + WriteManagedTypes (managedDuplicates); + + void WriteManagedTypes (IDictionary types) + { + var sortedTokens = types.Keys.ToArray (); + Array.Sort (sortedTokens); + + foreach (uint token in sortedTokens) { + bw.Write (token); + bw.Write (types[token]); + } + } + } + + void PadField (BinaryWriter bw, int valueWidth, int maxWidth) + { + for (int i = 0; i < maxWidth - valueWidth; i++) { + bw.Write ((byte)0); + } + } + } +} diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs index 5e1c82f1c5a..790fe071af5 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/TypeMappingNativeAssemblyGenerator.cs @@ -1,75 +1,239 @@ using System; +using System.Collections.Generic; using System.IO; -using Xamarin.Android.Tools; +using System.Linq; namespace Xamarin.Android.Tasks { class TypeMappingNativeAssemblyGenerator : NativeAssemblyGenerator { - NativeAssemblyDataStream dataStream; - string dataFileName; - uint dataSize; - string mappingFieldName; + readonly string baseFileName; + readonly NativeTypeMappingData mappingData; + readonly bool sharedBitsWritten; - public TypeMappingNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, NativeAssemblyDataStream dataStream, string dataFileName, uint dataSize, string mappingFieldName) - : base (targetProvider) + public TypeMappingNativeAssemblyGenerator (NativeAssemblerTargetProvider targetProvider, NativeTypeMappingData mappingData, string baseFileName, bool sharedBitsWritten, bool sharedIncludeUsesAbiPrefix = false) + : base (targetProvider, baseFileName, sharedIncludeUsesAbiPrefix) { - if (dataStream == null) - throw new ArgumentNullException (nameof (dataStream)); - if (String.IsNullOrEmpty (dataFileName)) - throw new ArgumentException ("must not be null or empty", nameof (dataFileName)); - if (String.IsNullOrEmpty (mappingFieldName)) - throw new ArgumentException ("must not be null or empty", nameof (mappingFieldName)); - - this.dataStream = dataStream; - this.dataFileName = dataFileName; - this.dataSize = dataSize; - this.mappingFieldName = mappingFieldName; + this.mappingData = mappingData ?? throw new ArgumentNullException (nameof (mappingData)); + + if (String.IsNullOrEmpty (baseFileName)) + throw new ArgumentException("must not be null or empty", nameof (baseFileName)); + + this.baseFileName = baseFileName; + this.sharedBitsWritten = sharedIncludeUsesAbiPrefix ? false : sharedBitsWritten; } - public bool EmbedAssemblies { get; set; } + protected override void WriteSymbols (StreamWriter output) + { + output.WriteLine (); + WriteHeaderField (output, "map_module_count", mappingData.MapModuleCount); + + output.WriteLine (); + WriteHeaderField (output, "java_type_count", mappingData.JavaTypeCount); + + output.WriteLine (); + WriteHeaderField (output, "java_name_width", mappingData.JavaNameWidth); + + bool haveAssemblyNames = mappingData.AssemblyNames.Count > 0; + bool haveModules = mappingData.Modules.Length > 0; - protected override void WriteFileHeader (StreamWriter output, string outputFileName) + output.WriteLine (); + if (haveAssemblyNames) { + output.WriteLine ($"{Indent}.include{Indent}\"{Path.GetFileName (SharedIncludeFile)}\""); + } else { + WriteCommentLine (output, $"No shared data present, {Path.GetFileName (SharedIncludeFile)} not generated"); + } + + if (haveModules) { + output.WriteLine ($"{Indent}.include{Indent}\"{Path.GetFileName (TypemapsIncludeFile)}\""); + } else { + WriteCommentLine (output, $"No modules defined, {Path.GetFileName (TypemapsIncludeFile)} not generated"); + } + output.WriteLine (); + + if (!sharedBitsWritten && haveAssemblyNames) { + using (var ms = new MemoryStream ()) { + using (var sharedOutput = new StreamWriter (ms, output.Encoding)) { + WriteAssemblyNames (sharedOutput); + sharedOutput.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, SharedIncludeFile); + } + } + } + + if (haveModules) { + using (var ms = new MemoryStream ()) { + using (var mapOutput = new StreamWriter (ms, output.Encoding)) { + WriteMapModules (output, mapOutput, "map_modules"); + mapOutput.Flush (); + MonoAndroidHelper.CopyIfStreamChanged (ms, TypemapsIncludeFile); + } + } + } else { + WriteMapModules (output, null, "map_modules"); + } + + WriteJavaMap (output, "map_java"); + } + + void WriteAssemblyNames (StreamWriter output) { - // The hash is written to make sure the assembly file which includes the data one is - // actually different whenever the data changes. Relying on mapping header values for this - // purpose would not be enough since the only change to the mapping might be a single-character - // change in one of the type names and we must be sure the assembly is rebuilt in all cases, - // thus the hash. - WriteEndLine (output, $"Data Hash: {Files.ToHexString (dataStream.GetStreamHash ())}", false); - base.WriteFileHeader (output, outputFileName); + foreach (var kvp in mappingData.AssemblyNames) { + string label = kvp.Key; + string name = kvp.Value; + + WriteData (output, name, label, isGlobal: false); + output.WriteLine (); + } } - protected override void WriteSymbols (StreamWriter output) + void WriteManagedMaps (StreamWriter output, string moduleSymbolName, IEnumerable entries) + { + if (entries == null) + return; + + var tokens = new Dictionary (); + foreach (TypeMapGenerator.TypeMapEntry entry in entries) { + int idx = Array.BinarySearch (mappingData.JavaTypeNames, entry.JavaName, StringComparer.Ordinal); + if (idx < 0) + throw new InvalidOperationException ($"Could not map entry '{entry.JavaName}' to array index"); + + tokens[entry.Token] = (uint)idx; + } + + WriteSection (output, $".rodata.{moduleSymbolName}", hasStrings: false, writable: false); + WriteStructureSymbol (output, moduleSymbolName, alignBits: 0, isGlobal: false); + + uint size = 0; + var sortedTokens = tokens.Keys.ToArray (); + Array.Sort (sortedTokens); + + foreach (uint token in sortedTokens) { + size += WriteStructure (output, packed: false, structureWriter: () => WriteManagedMapEntry (output, token, tokens[token])); + } + + WriteStructureSize (output, moduleSymbolName, size); + output.WriteLine (); + } + + uint WriteManagedMapEntry (StreamWriter output, uint token, uint javaMapIndex) { - WriteMappingHeader (output, dataStream, mappingFieldName); - WriteCommentLine (output, "Mapping data"); - WriteSymbol (output, mappingFieldName, dataSize, isGlobal: true, isObject: true, alwaysWriteSize: true); - if (EmbedAssemblies) { - output.WriteLine ($"{Indent}.include{Indent}\"{dataFileName}\""); + uint size = WriteData (output, token); + size += WriteData (output, javaMapIndex); + + return size; + } + + void WriteMapModules (StreamWriter output, StreamWriter mapOutput, string symbolName) + { + WriteCommentLine (output, "Managed to Java map: START", indent: false); + WriteSection (output, $".data.rel.{symbolName}", hasStrings: false, writable: true); + WriteStructureSymbol (output, symbolName, alignBits: TargetProvider.MapModulesAlignBits, isGlobal: true); + + uint size = 0; + int moduleCounter = 0; + foreach (TypeMapGenerator.ModuleData data in mappingData.Modules) { + string mapName = $"module{moduleCounter++}_managed_to_java"; + string duplicateMapName; + + if (data.DuplicateTypes.Count == 0) + duplicateMapName = null; + else + duplicateMapName = $"{mapName}_duplicates"; + + size += WriteStructure (output, packed: false, structureWriter: () => WriteMapModule (output, mapName, duplicateMapName, data)); + if (mapOutput != null) { + WriteManagedMaps (mapOutput, mapName, data.Types); + if (data.DuplicateTypes.Count > 0) + WriteManagedMaps (mapOutput, duplicateMapName, data.DuplicateTypes.Values); + } } + + WriteStructureSize (output, symbolName, size); + WriteCommentLine (output, "Managed to Java map: END", indent: false); + output.WriteLine (); } - void WriteMappingHeader (StreamWriter output, NativeAssemblyDataStream dataStream, string mappingFieldName) + uint WriteMapModule (StreamWriter output, string mapName, string duplicateMapName, TypeMapGenerator.ModuleData data) { + uint size = 0; + WriteCommentLine (output, $"module_uuid: {data.Mvid}"); + size += WriteData (output, data.MvidBytes); + + WriteCommentLine (output, "entry_count"); + size += WriteData (output, data.Types.Length); + + WriteCommentLine (output, "duplicate_count"); + size += WriteData (output, data.DuplicateTypes.Count); + + WriteCommentLine (output, "map"); + size += WritePointer (output, mapName); + + WriteCommentLine (output, "duplicate_map"); + size += WritePointer (output, duplicateMapName); + + WriteCommentLine (output, $"assembly_name: {data.AssemblyName}"); + size += WritePointer (output, MakeLocalLabel (data.AssemblyNameLabel)); + + WriteCommentLine (output, "image"); + size += WritePointer (output); + + // These two are used only in Debug builds with Instant Run enabled, but for simplicity we always output + // them. + WriteCommentLine (output, "java_name_width"); + size += WriteData (output, (uint)0); + + WriteCommentLine (output, "java_map"); + size += WritePointer (output); + output.WriteLine (); - WriteCommentLine (output, "Mapping header"); - WriteSection (output, $".data.{mappingFieldName}", hasStrings: false, writable: true); - WriteSymbol (output, $"{mappingFieldName}_header", alignBits: 2, fieldAlignBytes: 4, isGlobal: true, alwaysWriteSize: true, structureWriter: () => { - WriteCommentLine (output, "version"); - WriteData (output, dataStream.MapVersion); - - WriteCommentLine (output, "entry-count"); - WriteData (output, dataStream.MapEntryCount); - - WriteCommentLine (output, "entry-length"); - WriteData (output, dataStream.MapEntryLength); - - WriteCommentLine (output, "value-offset"); - WriteData (output, dataStream.MapValueOffset); - return 16; - }); + + return size; + } + + void WriteJavaMap (StreamWriter output, string symbolName) + { + WriteCommentLine (output, "Java to managed map: START", indent: false); + WriteSection (output, $".rodata.{symbolName}", hasStrings: false, writable: false); + WriteStructureSymbol (output, symbolName, alignBits: TargetProvider.MapJavaAlignBits, isGlobal: true); + + uint size = 0; + int entryCount = 0; + foreach (TypeMapGenerator.TypeMapEntry entry in mappingData.JavaTypes) { + size += WriteJavaMapEntry (output, entry, entryCount++); + } + + WriteStructureSize (output, symbolName, size); + WriteCommentLine (output, "Java to managed map: END", indent: false); + output.WriteLine (); + } + + uint WriteJavaMapEntry (StreamWriter output, TypeMapGenerator.TypeMapEntry entry, int entryIndex) + { + uint size = 0; + + WriteCommentLine (output, $"#{entryIndex}"); + WriteCommentLine (output, "module_index"); + size += WriteData (output, entry.ModuleIndex); + + WriteCommentLine (output, "type_token_id"); + size += WriteData (output, entry.Token); + + WriteCommentLine (output, "java_name"); + size += WriteAsciiData (output, entry.JavaName, mappingData.JavaNameWidth); + output.WriteLine (); + + return size; + } + + void WriteHeaderField (StreamWriter output, string name, uint value) + { + WriteCommentLine (output, $"{name}: START", indent: false); + WriteSection (output, $".rodata.{name}", hasStrings: false, writable: false); + WriteSymbol (output, name, size: 4, alignBits: 2, isGlobal: true, isObject: true, alwaysWriteSize: true); + WriteData (output, value); + WriteCommentLine (output, $"{name}: END", indent: false); } } } diff --git a/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs b/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs index f6f2a33144a..bb36f000526 100644 --- a/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs +++ b/src/Xamarin.Android.Build.Tasks/Utilities/X86NativeAssemblerTargetProvider.cs @@ -4,9 +4,15 @@ namespace Xamarin.Android.Tasks { class X86NativeAssemblerTargetProvider : NativeAssemblerTargetProvider { + const string X86 = "x86"; + const string X86_64 = "x86_64"; + public override bool Is64Bit { get; } public override string PointerFieldType { get; } public override string TypePrefix { get; } = "@"; + public override string AbiName => Is64Bit ? X86_64 : X86; + public override uint MapModulesAlignBits => Is64Bit ? 4u : 2u; + public override uint MapJavaAlignBits => Is64Bit ? 4u : 2u; public X86NativeAssemblerTargetProvider (bool is64Bit) { diff --git a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets index f8f2c88398f..38d83b6168a 100644 --- a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets +++ b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets @@ -351,7 +351,8 @@ Copyright (C) 2011-2012 Xamarin. All rights reserved. <_AndroidEnablePreloadAssembliesDefault>True $(_AndroidEnablePreloadAssembliesDefault) - <_NativeAssemblySourceDir>$(IntermediateOutputPath)android\ + <_NativeAssemblySourceDir>$(IntermediateOutputPath)android\ + <_AndroidUseNewTypemaps>True @@ -1162,8 +1163,6 @@ because xbuild doesn't support framework reference assemblies. <_RemoveRegisterFlag>$(MonoAndroidIntermediateAssemblyDir)shrunk\shrunk.flag <_AcwMapFile>$(IntermediateOutputPath)acw-map.txt <_CustomViewMapFile>$(IntermediateOutputPath)customview-map.txt - <_AndroidTypeMappingJavaToManaged>$(IntermediateOutputPath)android\typemap.jm - <_AndroidTypeMappingManagedToJava>$(IntermediateOutputPath)android\typemap.mj $(RootNamespace) <_AndroidLintConfigFile>$(IntermediateOutputPath)lint.xml <_AndroidResourceDesignerFile Condition=" '$(AndroidUseIntermediateDesignerFile)' == 'True' ">$(IntermediateOutputPath)$(_AndroidResourceDesigner) @@ -2039,23 +2038,19 @@ because xbuild doesn't support framework reference assemblies. - - - - - + BuildTargetAbis="@(_BuildTargetAbis)" + NativeSourcesDir="$(_NativeAssemblySourceDir)" + TypeMapMode="True"> + + <_GenerateJavaStubsDependsOnTargets> _SetLatestTargetFrameworkVersion; - _PrepareNativeAssemblySources; _PrepareAssemblies; + _PrepareNativeAssemblySources; $(_AfterPrepareAssemblies); @@ -2088,6 +2083,8 @@ because xbuild doesn't support framework reference assemblies. PackageName="$(_AndroidPackage)" ManifestPlaceholders="$(AndroidManifestPlaceholders)" OutputDirectory="$(IntermediateOutputPath)android" + TypemapOutputDirectory="$(_NativeAssemblySourceDir)" + GenerateNativeAssembly="!$(_InstantRunEnabled)" MergedAndroidManifestOutput="$(_ManifestOutput)" UseSharedRuntime="$(AndroidUseSharedRuntime)" EmbedAssemblies="$(EmbedAssembliesIntoApk)" @@ -2097,10 +2094,13 @@ because xbuild doesn't support framework reference assemblies. FrameworkDirectories="$(_XATargetFrameworkDirectories);$(_XATargetFrameworkDirectories)Facades" AcwMapFile="$(_AcwMapFile)" SupportedAbis="@(_BuildTargetAbis)"> + + + @@ -2157,8 +2157,9 @@ because xbuild doesn't support framework reference assemblies. - + NativeSourcesDir="$(_NativeAssemblySourceDir)" + TypeMapMode="false"> + @@ -2222,6 +2223,7 @@ because xbuild doesn't support framework reference assemblies. EnablePreloadAssembliesDefault="$(_AndroidEnablePreloadAssembliesDefault)" PackageNamingPolicy="$(AndroidPackageNamingPolicy)" BoundExceptionType="$(AndroidBoundExceptionType)" + InstantRunEnabled="$(_InstantRunEnabled)" > @@ -2535,7 +2537,6 @@ because xbuild doesn't support framework reference assemblies. @@ -2578,7 +2579,7 @@ because xbuild doesn't support framework reference assemblies. - + <_NativeAssemblyTarget Include="@(_TypeMapAssemblySource->Replace('.s', '.o'))"> %(_TypeMapAssemblySource.abi) @@ -2591,7 +2592,7 @@ because xbuild doesn't support framework reference assemblies. (fileStat.st_size), 1); - value = new char[r]; - size_t nread = fread (value, 1, static_cast(fileStat.st_size), fp); - if (nread == 0 || nread != static_cast(fileStat.st_size)) { - log_warn(LOG_DEFAULT, "While reading file %s: expected to read %u bytes, actually read %u bytes", path, r, nread); - } - } - fclose (fp); - } - return r; -} - #if defined (DEBUG) || !defined (ANDROID) size_t AndroidSystem::_monodroid_get_system_property_from_file (const char *path, char **value) diff --git a/src/monodroid/jni/android-system.hh b/src/monodroid/jni/android-system.hh index 6c6812a5982..13408ed3f2c 100644 --- a/src/monodroid/jni/android-system.hh +++ b/src/monodroid/jni/android-system.hh @@ -9,7 +9,7 @@ #include "util.hh" #include "cppcompat.hh" -#include "xamarin-app.h" +#include "xamarin-app.hh" #include "shared-constants.hh" #include "basic-android-system.hh" @@ -45,7 +45,6 @@ namespace xamarin::android::internal void create_update_dir (char *override_dir); int monodroid_get_system_property (const char *name, char **value); size_t monodroid_get_system_property_from_overrides (const char *name, char ** value); - size_t monodroid_read_file_into_memory (const char *path, char *&value); char* get_bundled_app (JNIEnv *env, jstring dir); int count_override_assemblies (); long get_gref_gc_threshold (); diff --git a/src/monodroid/jni/application_dso_stub.cc b/src/monodroid/jni/application_dso_stub.cc index 459bb600391..2cc3c860551 100644 --- a/src/monodroid/jni/application_dso_stub.cc +++ b/src/monodroid/jni/application_dso_stub.cc @@ -1,16 +1,16 @@ #include #include -#include "xamarin-app.h" +#include "xamarin-app.hh" -// This file MUST have valid values everywhere - the DSO it is compiled into is loaded by the +// This file MUST have "valid" values everywhere - the DSO it is compiled into is loaded by the // designer on desktop. +const uint32_t map_module_count = 0; +const uint32_t java_type_count = 0; +const uint32_t java_name_width = 0; -TypeMapHeader jm_typemap_header = { 1, 0, 0, 0 }; -uint8_t jm_typemap[] = { 0 }; - -TypeMapHeader mj_typemap_header = { 1, 0, 0, 0 }; -uint8_t mj_typemap[] = { 0 }; +const TypeMapModule map_modules[] = {}; +const TypeMapJava map_java[] = {}; ApplicationConfig application_config = { .uses_mono_llvm = false, diff --git a/src/monodroid/jni/cpp-util.hh b/src/monodroid/jni/cpp-util.hh index 686c86da5b7..7195f2c33c0 100644 --- a/src/monodroid/jni/cpp-util.hh +++ b/src/monodroid/jni/cpp-util.hh @@ -51,17 +51,6 @@ namespace xamarin::android return tp; } - protected: - void set_pointer (T* ptr) noexcept - { - tp = ptr; - } - - void set_pointer (const T* ptr) noexcept - { - tp = ptr; - } - private: T *tp = nullptr; }; diff --git a/src/monodroid/jni/embedded-assemblies-zip.cc b/src/monodroid/jni/embedded-assemblies-zip.cc index 8755a65c6bb..fa5e0193d04 100644 --- a/src/monodroid/jni/embedded-assemblies-zip.cc +++ b/src/monodroid/jni/embedded-assemblies-zip.cc @@ -74,19 +74,6 @@ EmbeddedAssemblies::zip_load_entries (int fd, const char *apk_name, monodroid_sh if (compression_method != 0) continue; -#if defined (DEBUG) - if (utils.ends_with (file_name, ".jm")) { - md_mmap_info map_info = md_mmap_apk_file (fd, data_offset, file_size, file_name, apk_name); - add_type_mapping (&java_to_managed_maps, apk_name, file_name, (const char*)map_info.area); - continue; - } - - if (utils.ends_with (file_name, ".mj")) { - md_mmap_info map_info = md_mmap_apk_file (fd, data_offset, file_size, file_name, apk_name); - add_type_mapping (&managed_to_java_maps, apk_name, file_name, (const char*)map_info.area); - continue; - } -#endif if (strncmp (prefix, file_name, prefix_len) != 0) continue; diff --git a/src/monodroid/jni/embedded-assemblies.cc b/src/monodroid/jni/embedded-assemblies.cc index 9c703e8c70d..19f4467a544 100644 --- a/src/monodroid/jni/embedded-assemblies.cc +++ b/src/monodroid/jni/embedded-assemblies.cc @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -22,26 +23,36 @@ #include "embedded-assemblies.hh" #include "globals.hh" #include "monodroid-glue.hh" -#include "xamarin-app.h" +#include "xamarin-app.hh" #include "cpp-util.hh" -namespace xamarin::android::internal { -#if defined (DEBUG) || !defined (ANDROID) - struct TypeMappingInfo { - char *source_apk; - char *source_entry; - int num_entries; - int entry_length; - int value_offset; - const char *mapping; - TypeMappingInfo *next; - }; -#endif // DEBUG || !ANDROID -} - using namespace xamarin::android; using namespace xamarin::android::internal; +// A utility class which allows us to manage memory allocated by `mono_guid_to_string` in an elegant way. We can create +// temporary instances of this class in calls to e.g. `log_debug` which are executed ONLY when debug logging is enabled +class MonoGuidString +{ +public: + explicit MonoGuidString (const uint8_t *id) noexcept + { + guid = mono_guid_to_string (id); + } + + ~MonoGuidString () + { + ::free (guid); + } + + const char* get () const noexcept + { + return guid; + } + +private: + char *guid = nullptr; +}; + void EmbeddedAssemblies::set_assemblies_prefix (const char *prefix) { if (assemblies_prefix_override != nullptr) @@ -119,120 +130,284 @@ EmbeddedAssemblies::install_preload_hooks () mono_install_assembly_refonly_preload_hook (open_from_bundles_refonly, nullptr); } -int -EmbeddedAssemblies::TypeMappingInfo_compare_key (const void *a, const void *b) +template +const Entry* +EmbeddedAssemblies::binary_search (const Key *key, const Entry *base, size_t nmemb, [[maybe_unused]] size_t extra_size) { - return strcmp (reinterpret_cast (a), reinterpret_cast (b)); -} + static_assert (compare != nullptr, "compare is a required template parameter"); -inline const char* -EmbeddedAssemblies::find_entry_in_type_map (const char *name, uint8_t map[], TypeMapHeader& header) -{ - const char *e = reinterpret_cast (bsearch (name, map, header.entry_count, header.entry_length, TypeMappingInfo_compare_key )); - if (e == nullptr) + // This comes from the user code, so let's be civil + if (key == nullptr) { + log_warn (LOG_ASSEMBLY, "Key passed to binary_search must not be nullptr"); return nullptr; - return e + header.value_offset; + } + + // This is a coding error on our part, crash! + if (base == nullptr) { + log_fatal (LOG_ASSEMBLY, "Map address not passed to binary_search"); + exit (FATAL_EXIT_MISSING_ASSEMBLY); + } + + constexpr size_t size = sizeof(Entry); + while (nmemb > 0) { + const Entry *ret; + if constexpr (use_extra_size) { + ret = reinterpret_cast(reinterpret_cast(base) + ((size + extra_size) * (nmemb / 2))); + } else { + ret = base + (nmemb / 2); + } + + int result = compare (key, ret); + if (result < 0) { + nmemb /= 2; + } else if (result > 0) { + if constexpr (use_extra_size) { + base = reinterpret_cast(reinterpret_cast(ret) + size + extra_size); + } else { + base = ret + 1; + } + nmemb -= nmemb / 2 + 1; + } else { + return ret; + } + } + + return nullptr; } -const char* -EmbeddedAssemblies::typemap_java_to_managed (const char *java) +MonoReflectionType* +EmbeddedAssemblies::typemap_java_to_managed (MonoString *java_type) { + timing_period total_time; + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + timing = new Timing (); + total_time.mark_start (); + } + + if (XA_UNLIKELY (java_type == nullptr)) { + log_warn (LOG_ASSEMBLY, "typemap: null 'java_type' passed to 'typemap_java_to_managed'"); + return nullptr; + } + + simple_pointer_guard java_type_name (mono_string_to_utf8 (java_type)); + if (XA_UNLIKELY (!java_type_name || *java_type_name == '\0')) { + log_warn (LOG_ASSEMBLY, "typemap: empty Java type name passed to 'typemap_java_to_managed'"); + return nullptr; + } + + int32_t type_token_id = -1; + TypeMapModule *module; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + size_t idx = 0; + for (; idx < module_count; idx++) { + const uint8_t *java_entry = binary_search (java_type_name.get (), modules[idx].java_map, modules[idx].entry_count, modules[idx].java_name_width + 3); + if (java_entry == nullptr) + continue; + type_token_id = *reinterpret_cast(java_entry + modules[idx].java_name_width); + break; + } + + if (idx >= module_count) { + log_error (LOG_ASSEMBLY, "typemap: unable to find module with Java type '%s' mapping", java_type_name.get ()); + return nullptr; + } + + module = &modules[idx]; + } else { +#endif + const TypeMapJava *java_entry = binary_search (java_type_name.get (), map_java, java_type_count, java_name_width); + if (java_entry == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: unable to find mapping to a managed type from Java type '%s'", java_type_name.get ()); + return nullptr; + } + + if (java_entry->module_index >= map_module_count) { + log_warn (LOG_ASSEMBLY, "typemap: mapping from Java type '%s' to managed type has invalid module index", java_type_name.get ()); + return nullptr; + } + + module = const_cast(&map_modules[java_entry->module_index]); + const TypeMapModuleEntry *entry = binary_search (&java_entry->type_token_id, module->map, module->entry_count); + if (entry == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: unable to find mapping from Java type '%s' to managed type with token ID %u in module [%s]", java_type_name.get (), java_entry->type_token_id, MonoGuidString (module->module_uuid).get ()); + return nullptr; + } + type_token_id = java_entry->type_token_id; #if defined (DEBUG) || !defined (ANDROID) - for (TypeMappingInfo *info = java_to_managed_maps; info != nullptr; info = info->next) { - /* log_warn (LOG_DEFAULT, "# jonp: checking file: %s!%s for type '%s'", info->source_apk, info->source_entry, java); */ - const char *e = reinterpret_cast (bsearch (java, info->mapping, static_cast(info->num_entries), static_cast(info->entry_length), TypeMappingInfo_compare_key)); - if (e == nullptr) - continue; - return e + info->value_offset; } #endif - return find_entry_in_type_map (java, jm_typemap, jm_typemap_header); + + if (module->image == nullptr) { + module->image = mono_image_loaded (module->assembly_name); + if (module->image == nullptr) { + // TODO: load + log_error (LOG_ASSEMBLY, "typemap: assembly '%s' not loaded yet!", module->assembly_name); + } + + if (module->image == nullptr) { + log_error (LOG_ASSEMBLY, "typemap: unable to load assembly '%s' when looking up managed type corresponding to Java type '%s'", module->assembly_name, java_type_name.get ()); + return nullptr; + } + } + + log_debug (LOG_ASSEMBLY, "typemap: java type '%s' corresponds to managed token id %u (0x%x)", java_type_name.get (), type_token_id, type_token_id); + MonoClass *klass = mono_class_get (module->image, static_cast(type_token_id)); + if (klass == nullptr) { + log_error (LOG_ASSEMBLY, "typemap: unable to find managed type with token ID %u in assembly '%s', corresponding to Java type '%s'", type_token_id, module->assembly_name, java_type_name.get ()); + return nullptr; + } + + MonoReflectionType *ret = mono_type_get_object (mono_domain_get (), mono_class_get_type (klass)); + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + total_time.mark_end (); + + Timing::info (total_time, "Typemap.java_to_managed: end, total time"); + } + + return ret; } -const char* -EmbeddedAssemblies::typemap_managed_to_java (const char *managed) +int +EmbeddedAssemblies::compare_java_name (const char *java_name, const TypeMapJava *entry) { -#if defined (DEBUG) || !defined (ANDROID) - for (TypeMappingInfo *info = managed_to_java_maps; info != nullptr; info = info->next) { - /* log_warn (LOG_DEFAULT, "# jonp: checking file: %s!%s for type '%s'", info->source_apk, info->source_entry, managed); */ - const char *e = reinterpret_cast (bsearch (managed, info->mapping, static_cast(info->num_entries), static_cast(info->entry_length), TypeMappingInfo_compare_key)); - if (e == nullptr) - continue; - return e + info->value_offset; + if (entry == nullptr || entry->java_name[0] == '\0') { + return -1; } -#endif - return find_entry_in_type_map (managed, mj_typemap, mj_typemap_header); + + return strcmp (java_name, reinterpret_cast(entry->java_name)); } #if defined (DEBUG) || !defined (ANDROID) -void -EmbeddedAssemblies::extract_int (const char **header, const char *source_apk, const char *source_entry, const char *key_name, int *value) +int +EmbeddedAssemblies::compare_java_name (const char *java_name, const uint8_t *entry) { - int read = 0; - int consumed = 0; - size_t key_name_len = 0; - char scanf_format [20] = { 0, }; + if (entry == nullptr) + return 1; - if (header == nullptr || *header == nullptr) - return; + return strcmp (java_name, reinterpret_cast(entry)); +} +#endif // DEBUG || !ANDROID - key_name_len = strlen (key_name); - if (key_name_len >= (sizeof (scanf_format) - sizeof ("=%d%n"))) { - *header = nullptr; - return; +const char* +EmbeddedAssemblies::typemap_managed_to_java (const uint8_t *mvid, const int32_t token) +{ + timing_period total_time; + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + timing = new Timing (); + total_time.mark_start (); } - snprintf (scanf_format, sizeof (scanf_format), "%s=%%d%%n", key_name); + if (mvid == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: no mvid specified in call to typemap_managed_to_java"); + return nullptr; + } - read = sscanf (*header, scanf_format, value, &consumed); - if (read != 1) { - log_warn (LOG_DEFAULT, "Could not read header '%s' value from '%s!%s': read %i elements, expected 1 element. Contents: '%s'", - key_name, source_apk, source_entry, read, *header); - *header = nullptr; - return; + const TypeMapModule *map; + size_t map_entry_count; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + map = modules; + map_entry_count = module_count; + } else { +#endif + map = map_modules; + map_entry_count = map_module_count; +#if defined (DEBUG) || !defined (ANDROID) + } +#endif + const TypeMapModule *match = binary_search (mvid, map, map_entry_count); + if (match == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: module matching MVID [%s] not found.", MonoGuidString (mvid).get ()); + return nullptr; } - *header = *header + consumed + 1; -} -bool -EmbeddedAssemblies::add_type_mapping (TypeMappingInfo **info, const char *source_apk, const char *source_entry, const char *addr) -{ - TypeMappingInfo *p = new TypeMappingInfo (); // calloc (1, sizeof (struct TypeMappingInfo)); - int version = 0; - const char *data = addr; - - extract_int (&data, source_apk, source_entry, "version", &version); - if (version != 1) { - delete p; - log_warn (LOG_DEFAULT, "Unsupported version '%i' within type mapping file '%s!%s'. Ignoring...", version, source_apk, source_entry); - return false; + if (match->map == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: module with MVID [%s] has no associated type map.", MonoGuidString (mvid).get ()); + return nullptr; } - extract_int (&data, source_apk, source_entry, "entry-count", &p->num_entries); - extract_int (&data, source_apk, source_entry, "entry-len", &p->entry_length); - extract_int (&data, source_apk, source_entry, "value-offset", &p->value_offset); - p->mapping = data; + log_debug (LOG_ASSEMBLY, "typemap: MVID [%s] maps to assembly %s, looking for token %d (0x%x), table index %d", MonoGuidString (mvid).get (), match->assembly_name, token, token, token & 0x00FFFFFF); + // Each map entry is a pair of 32-bit integers: [TypeTokenID][JavaMapArrayIndex] + const TypeMapModuleEntry *entry = binary_search (&token, match->map, match->entry_count); + if (entry == nullptr) { + if (match->duplicate_count > 0 && match->duplicate_map != nullptr) { + log_debug (LOG_ASSEMBLY, "typemap: searching module [%s] duplicate map for token %u (0x%x)", MonoGuidString (mvid).get (), token, token); + entry = binary_search (&token, match->duplicate_map, match->duplicate_count); + } - if ((p->mapping == 0) || - (p->num_entries <= 0) || - (p->entry_length <= 0) || - (p->value_offset >= p->entry_length) || - (p->mapping == nullptr)) { - log_warn (LOG_DEFAULT, "Could not read type mapping file '%s!%s'. Ignoring...", source_apk, source_entry); - delete p; - return false; + if (entry == nullptr) { + log_warn (LOG_ASSEMBLY, "typemap: type with token %d (0x%x) in module {%s} (%s) not found.", token, token, MonoGuidString (mvid).get (), match->assembly_name); + return nullptr; + } } - p->source_apk = strdup (source_apk); - p->source_entry = strdup (source_entry); - if (*info) { - (*info)->next = p; + uint32_t java_entry_count; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + java_entry_count = match->entry_count; } else { - *info = p; +#endif + java_entry_count = java_type_count; +#if defined (DEBUG) || !defined (ANDROID) } - return true; +#endif + if (entry->java_map_index >= java_entry_count) { + log_warn (LOG_ASSEMBLY, "typemap: type with token %d (0x%x) in module {%s} (%s) has invalid Java type index %u", token, token, MonoGuidString (mvid).get (), match->assembly_name, entry->java_map_index); + return nullptr; + } + + const char *ret; +#if defined (DEBUG) || !defined (ANDROID) + if (application_config.instant_run_enabled) { + ret = reinterpret_cast(match->java_map + ((match->java_name_width + 4) * entry->java_map_index)); + } else { +#endif + const TypeMapJava *java_entry = reinterpret_cast (reinterpret_cast(map_java) + ((sizeof(TypeMapJava) + java_name_width) * entry->java_map_index)); + ret = reinterpret_cast(reinterpret_cast(java_entry) + 8); +#if defined (DEBUG) || !defined (ANDROID) + } +#endif + + if (XA_UNLIKELY (ret == nullptr)) { + log_warn (LOG_ASSEMBLY, "typemap: empty Java type name returned for entry at index %u", entry->java_map_index); + } + + if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) { + total_time.mark_end (); + + Timing::info (total_time, "Typemap.managed_to_java: end, total time"); + } + + log_debug ( + LOG_ASSEMBLY, + "typemap: type with token %d (0x%x) in module {%s} (%s) corresponds to Java type '%s'", + token, + token, + MonoGuidString (mvid).get (), + match->assembly_name, + ret + ); + + return ret; +} + +int +EmbeddedAssemblies::compare_type_token (const int32_t *token, const TypeMapModuleEntry *entry) +{ + if (entry == nullptr) { + log_fatal (LOG_ASSEMBLY, "typemap: compare_type_token: entry is nullptr"); + exit (FATAL_EXIT_MISSING_ASSEMBLY); + } + + return *token - entry->type_token_id; +} + +int +EmbeddedAssemblies::compare_mvid (const uint8_t *mvid, const TypeMapModule *module) +{ + return memcmp (mvid, module->module_uuid, sizeof(module->module_uuid)); } -#endif // DEBUG || !ANDROID EmbeddedAssemblies::md_mmap_info EmbeddedAssemblies::md_mmap_apk_file (int fd, uint32_t offset, uint32_t size, const char* filename, const char* apk) @@ -299,50 +474,270 @@ EmbeddedAssemblies::gather_bundled_assemblies_from_apk (const char* apk, monodro } #if defined (DEBUG) || !defined (ANDROID) +ssize_t EmbeddedAssemblies::do_read (int fd, void *buf, size_t count) +{ + ssize_t ret; + do { + ret = ::read (fd, buf, count); + } while (ret < 0 && errno == EINTR); + + return ret; +} + +template +bool +EmbeddedAssemblies::typemap_read_header ([[maybe_unused]] int dir_fd, const char *file_type, const char *dir_path, const char *file_path, uint32_t expected_magic, H &header, size_t &file_size, int &fd) +{ + struct stat sbuf; + int res; + +#if __ANDROID_API__ < 21 + simple_pointer_guard full_file_path = utils.path_combine (dir_path, file_path); + res = stat (full_file_path, &sbuf); +#else + res = fstatat (dir_fd, file_path, &sbuf, 0); +#endif + if (res < 0) { + log_error (LOG_ASSEMBLY, "typemap: failed to stat %s file '%s/%s': %s", file_type, dir_path, file_path, strerror (errno)); + return false; + } + + file_size = static_cast(sbuf.st_size); + if (file_size < sizeof (header)) { + log_error (LOG_ASSEMBLY, "typemap: %s file '%s/%s' is too small (must be at least %u bytes)", file_type, dir_path, file_path, sizeof (header)); + return false; + } + +#if __ANDROID_API__ < 21 + fd = open (full_file_path, O_RDONLY); +#else + fd = openat (dir_fd, file_path, O_RDONLY); +#endif + if (fd < 0) { + log_error (LOG_ASSEMBLY, "typemap: failed to open %s file %s/%s for reading: %s", file_type, dir_path, file_path, strerror (errno)); + return false; + } + + ssize_t nread = do_read (fd, &header, sizeof (header)); + if (nread <= 0) { + if (nread < 0) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %s file header from '%s/%s': %s", file_type, dir_path, file_path, strerror (errno)); + } else { + log_error (LOG_ASSEMBLY, "typemap: end of file while reading %s file header from '%s/%s'", file_type, dir_path, file_path); + } + + return false; + } + + if (header.magic != expected_magic) { + log_error (LOG_ASSEMBLY, "typemap: invalid magic value in the %s file header from '%s/%s': expected 0x%X, got 0x%X", file_type, dir_path, file_path, expected_magic, header.magic); + return false; + } + + if (header.version != MODULE_FORMAT_VERSION) { + log_error (LOG_ASSEMBLY, "typemap: incompatible %s format version. This build supports only version %u, file '%s/%s' uses version %u", file_type, MODULE_FORMAT_VERSION, dir_path, file_path, header.version); + return false; + } + + return true; +} + +uint8_t* +EmbeddedAssemblies::typemap_load_index (TypeMapIndexHeader &header, size_t file_size, int index_fd) +{ + constexpr size_t UUID_SIZE = 16; + + size_t entry_size = header.module_file_name_width + UUID_SIZE; + size_t data_size = entry_size * module_count; + if (sizeof(header) + data_size > file_size) { + log_error (LOG_ASSEMBLY, "typemap: index file is too small, expected %u, found %u bytes", data_size + sizeof(header), file_size); + return nullptr; + } + + auto data = new uint8_t [data_size]; + ssize_t nread = do_read (index_fd, data, data_size); + if (nread != static_cast(data_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes from index file. %s", data_size, strerror (errno)); + return nullptr; + } + + uint8_t *p = data; + for (size_t i = 0; i < module_count; i++) { + memcpy (modules[i].module_uuid, p, UUID_SIZE); + modules[i].assembly_name = reinterpret_cast(p + UUID_SIZE); + p += entry_size; + } + + return data; +} + +uint8_t* +EmbeddedAssemblies::typemap_load_index (int dir_fd, const char *dir_path, const char *index_path) +{ + log_debug (LOG_ASSEMBLY, "typemap: loading TypeMap index file '%s/%s'", dir_path, index_path); + + TypeMapIndexHeader header; + size_t file_size; + int fd = -1; + uint8_t *data = nullptr; + + if (!typemap_read_header (dir_fd, "TypeMap index", dir_path, index_path, MODULE_INDEX_MAGIC, header, file_size, fd)) { + goto cleanup; + } + + module_count = header.entry_count; + modules = new TypeMapModule[module_count](); + data = typemap_load_index (header, file_size, fd); + + cleanup: + if (fd >= 0) + close (fd); + + return data; +} + +bool +EmbeddedAssemblies::typemap_load_file (BinaryTypeMapHeader &header, const char *dir_path, const char *file_path, int file_fd, TypeMapModule &module) +{ + size_t alloc_size = ADD_WITH_OVERFLOW_CHECK (size_t, header.assembly_name_length, 1); + module.assembly_name = new char[alloc_size]; + + ssize_t nread = do_read (file_fd, module.assembly_name, header.assembly_name_length); + module.assembly_name [header.assembly_name_length] = 0; + module.entry_count = header.entry_count; + + log_debug ( + LOG_ASSEMBLY, + "typemap: '%s/%s':: entry count == %u; duplicate entry count == %u; Java type name field width == %u; MVID == %s; assembly name length == %u; assembly name == %s", + dir_path, file_path, header.entry_count, header.duplicate_count, header.java_name_width, + MonoGuidString (header.module_uuid).get (), header.assembly_name_length, module.assembly_name + ); + + alloc_size = MULTIPLY_WITH_OVERFLOW_CHECK (size_t, header.java_name_width + 4, header.entry_count); + module.java_name_width = header.java_name_width; + module.java_map = new uint8_t[alloc_size]; + nread = do_read (file_fd, module.java_map, alloc_size); + if (nread != static_cast(alloc_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes (java-to-managed) from module file %s/%s. %s", alloc_size, dir_path, file_path, strerror (errno)); + return false; + } + + module.map = new TypeMapModuleEntry[header.entry_count]; + alloc_size = MULTIPLY_WITH_OVERFLOW_CHECK (size_t, sizeof(TypeMapModuleEntry), header.entry_count); + nread = do_read (file_fd, module.map, alloc_size); + if (nread != static_cast(alloc_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes (managed-to-java) from module file %s/%s. %s", alloc_size, dir_path, file_path, strerror (errno)); + return false; + } + + // alloc_size = module.java_name_width + 1; + // auto chars = new char[alloc_size](); + // uint8_t *p = module.java_map; + // log_debug (LOG_ASSEMBLY, "Java entries in %s/%s", dir_path, file_path); + // for (size_t i = 0; i < module.entry_count; i++) { + // memcpy (chars, p, module.java_name_width); + // uint32_t token = *reinterpret_cast(p + module.java_name_width); + // log_debug (LOG_ASSEMBLY, " %04u: %s; %u (0x%x)", i, chars, token, token); + // p += module.java_name_width + 4; + // } + // delete[] chars; + + // log_debug (LOG_ASSEMBLY, "Managed entries in %s/%s", dir_path, file_path); + // for (size_t i = 0; i < module.entry_count; i++) { + // log_debug (LOG_ASSEMBLY, " %04u: token %u (0x%x); index %u", i, module.map[i].type_token_id, module.map[i].type_token_id, module.map[i].java_map_index); + // } + + if (header.duplicate_count == 0) + return true; + + module.duplicate_map = new TypeMapModuleEntry[header.duplicate_count]; + alloc_size = MULTIPLY_WITH_OVERFLOW_CHECK (size_t, sizeof(TypeMapModuleEntry), header.duplicate_count); + nread = do_read (file_fd, module.duplicate_map, alloc_size); + if (nread != static_cast(alloc_size)) { + log_error (LOG_ASSEMBLY, "typemap: failed to read %u bytes (managed-to-java duplicates) from module file %s/%s. %s", alloc_size, dir_path, file_path, strerror (errno)); + return false; + } + + return true; +} + +bool +EmbeddedAssemblies::typemap_load_file (int dir_fd, const char *dir_path, const char *file_path, TypeMapModule &module) +{ + log_debug (LOG_ASSEMBLY, "typemap: loading TypeMap file '%s/%s'", dir_path, file_path); + + bool ret = true; + BinaryTypeMapHeader header; + size_t file_size; + int fd = -1; + + module.java_map = nullptr; + module.map = nullptr; + module.duplicate_map = nullptr; + + if (!typemap_read_header (dir_fd, "TypeMap", dir_path, file_path, MODULE_MAGIC, header, file_size, fd)) { + ret = false; + goto cleanup; + } + + ret = typemap_load_file (header, dir_path, file_path, fd, module); + + cleanup: + if (fd >= 0) + close (fd); + + if (!ret) { + delete[] module.java_map; + module.java_map = nullptr; + delete[] module.map; + module.map = nullptr; + delete[] module.duplicate_map; + module.duplicate_map = nullptr; + } + + return ret; +} + void EmbeddedAssemblies::try_load_typemaps_from_directory (const char *path) { - // read the entire typemap file into a string - // process the string using the add_type_mapping - char *dir_path = utils.path_combine (path, "typemaps"); - if (dir_path == nullptr || !utils.directory_exists (dir_path)) { - log_warn (LOG_DEFAULT, "directory does not exist: `%s`", dir_path); - free (dir_path); + if (!application_config.instant_run_enabled) { + log_info (LOG_ASSEMBLY, "typemap: instant run disabled, not loading type maps from storage"); return; } + simple_pointer_guard dir_path (utils.path_combine (path, "typemaps")); monodroid_dir_t *dir; if ((dir = utils.monodroid_opendir (dir_path)) == nullptr) { - log_warn (LOG_DEFAULT, "could not open directory: `%s`", dir_path); - free (dir_path); + log_warn (LOG_DEFAULT, "typemap: could not open directory: `%s`", dir_path.get ()); return; } - monodroid_dirent_t *e; - while ((e = androidSystem.readdir (dir)) != nullptr) { -#if WINDOWS - char *file_name = utils.utf16_to_utf8 (e->d_name); -#else /* def WINDOWS */ - char *file_name = e->d_name; -#endif /* ndef WINDOWS */ - char *file_path = utils.path_combine (dir_path, file_name); - if (utils.monodroid_dirent_hasextension (e, ".mj") || utils.monodroid_dirent_hasextension (e, ".jm")) { - char *val = nullptr; - size_t len = androidSystem.monodroid_read_file_into_memory (file_path, val); - if (len > 0 && val != nullptr) { - if (utils.monodroid_dirent_hasextension (e, ".mj")) { - if (!add_type_mapping (&managed_to_java_maps, file_path, override_typemap_entry_name, ((const char*)val))) - delete[] val; - } else if (utils.monodroid_dirent_hasextension (e, ".jm")) { - if (!add_type_mapping (&java_to_managed_maps, file_path, override_typemap_entry_name, ((const char*)val))) - delete[] val; - } - } + int dir_fd; +#if __ANDROID_API__ < 21 + dir_fd = -1; +#else + dir_fd = dirfd (dir); +#endif + + constexpr char index_name[] = "typemap.index"; + + // The pointer must be stored here because, after index is loaded, module.assembly_name points into the index data + // and must be valid until after the actual module file is loaded. + simple_pointer_guard index_data = typemap_load_index (dir_fd, dir_path, index_name); + if (!index_data) { + log_fatal (LOG_ASSEMBLY, "typemap: unable to load TypeMap data index from '%s/%s'", dir_path.get (), index_name); + exit (FATAL_EXIT_NO_ASSEMBLIES); // TODO: use a new error code here + } + + for (size_t i = 0; i < module_count; i++) { + TypeMapModule &module = modules[i]; + if (!typemap_load_file (dir_fd, dir_path, module.assembly_name, module)) { + continue; } } + utils.monodroid_closedir (dir); - free (dir_path); - return; } #endif diff --git a/src/monodroid/jni/embedded-assemblies.hh b/src/monodroid/jni/embedded-assemblies.hh index 1f3a024c960..f000bab413b 100644 --- a/src/monodroid/jni/embedded-assemblies.hh +++ b/src/monodroid/jni/embedded-assemblies.hh @@ -3,8 +3,11 @@ #define INC_MONODROID_EMBEDDED_ASSEMBLIES_H #include +#include #include +#include "xamarin-app.hh" + struct TypeMapHeader; namespace xamarin::android::internal { @@ -39,8 +42,8 @@ namespace xamarin::android::internal { void try_load_typemaps_from_directory (const char *path); #endif void install_preload_hooks (); - const char* typemap_java_to_managed (const char *java); - const char* typemap_managed_to_java (const char *managed); + MonoReflectionType* typemap_java_to_managed (MonoString *java_type); + const char* typemap_managed_to_java (const uint8_t *mvid, const int32_t token); /* returns current number of *all* assemblies found from all invocations */ template @@ -66,9 +69,14 @@ namespace xamarin::android::internal { size_t register_from (const char *apk_file, monodroid_should_register should_register); void gather_bundled_assemblies_from_apk (const char* apk, monodroid_should_register should_register); MonoAssembly* open_from_bundles (MonoAssemblyName* aname, bool ref_only); - void extract_int (const char **header, const char *source_apk, const char *source_entry, const char *key_name, int *value); #if defined (DEBUG) || !defined (ANDROID) - bool add_type_mapping (TypeMappingInfo **info, const char *source_apk, const char *source_entry, const char *addr); + template + bool typemap_read_header (int dir_fd, const char *file_type, const char *dir_path, const char *file_path, uint32_t expected_magic, H &header, size_t &file_size, int &fd); + uint8_t* typemap_load_index (int dir_fd, const char *dir_path, const char *index_path); + uint8_t* typemap_load_index (TypeMapIndexHeader &header, size_t file_size, int index_fd); + bool typemap_load_file (int dir_fd, const char *dir_path, const char *file_path, TypeMapModule &module); + bool typemap_load_file (BinaryTypeMapHeader &header, const char *dir_path, const char *file_path, int file_fd, TypeMapModule &module); + static ssize_t do_read (int fd, void *buf, size_t count); #endif // DEBUG || !ANDROID bool register_debug_symbols_for_assembly (const char *entry_name, MonoBundledAssembly *assembly, const mono_byte *debug_contents, int debug_size); @@ -76,8 +84,6 @@ namespace xamarin::android::internal { static MonoAssembly* open_from_bundles_full (MonoAssemblyName *aname, char **assemblies_path, void *user_data); static MonoAssembly* open_from_bundles_refonly (MonoAssemblyName *aname, char **assemblies_path, void *user_data); - static int TypeMappingInfo_compare_key (const void *a, const void *b); - const char *find_entry_in_type_map (const char *name, uint8_t map[], TypeMapHeader& header); void zip_load_entries (int fd, const char *apk_name, monodroid_should_register should_register); bool zip_read_cd_info (int fd, uint32_t& cd_offset, uint32_t& cd_size, uint16_t& cd_entries); @@ -95,6 +101,16 @@ namespace xamarin::android::internal { return assemblies_prefix_override != nullptr ? assemblies_prefix_override : assemblies_prefix; } + template + const Entry* binary_search (const Key *key, const Entry *base, size_t nmemb, size_t extra_size = 0); + + static int compare_mvid (const uint8_t *mvid, const TypeMapModule *module); + static int compare_type_token (const int32_t *token, const TypeMapModuleEntry *entry); + static int compare_java_name (const char *java_name, const TypeMapJava *entry); +#if defined (DEBUG) || !defined (ANDROID) + static int compare_java_name (const char *java_name, const uint8_t *java_map); +#endif + private: bool register_debug_symbols; MonoBundledAssembly **bundled_assemblies = nullptr; @@ -102,6 +118,8 @@ namespace xamarin::android::internal { #if defined (DEBUG) || !defined (ANDROID) TypeMappingInfo *java_to_managed_maps; TypeMappingInfo *managed_to_java_maps; + TypeMapModule *modules; + size_t module_count; #endif // DEBUG || !ANDROID const char *assemblies_prefix_override = nullptr; }; diff --git a/src/monodroid/jni/external-api.cc b/src/monodroid/jni/external-api.cc index 49e24da3d79..c49a58308af 100644 --- a/src/monodroid/jni/external-api.cc +++ b/src/monodroid/jni/external-api.cc @@ -159,15 +159,9 @@ _monodroid_get_display_dpi (float *x_dpi, float *y_dpi) } MONO_API const char * -monodroid_typemap_java_to_managed (const char *java) +monodroid_typemap_managed_to_java (const uint8_t *mvid, const int32_t token) { - return embeddedAssemblies.typemap_java_to_managed (java); -} - -MONO_API const char * -monodroid_typemap_managed_to_java (const char *managed) -{ - return embeddedAssemblies.typemap_managed_to_java (managed); + return embeddedAssemblies.typemap_managed_to_java (mvid, token); } MONO_API int monodroid_embedded_assemblies_set_assemblies_prefix (const char *prefix) diff --git a/src/monodroid/jni/monodroid-glue-internal.hh b/src/monodroid/jni/monodroid-glue-internal.hh index 390b97d815a..c4afbfc8e1a 100644 --- a/src/monodroid/jni/monodroid-glue-internal.hh +++ b/src/monodroid/jni/monodroid-glue-internal.hh @@ -175,6 +175,7 @@ namespace xamarin::android::internal static void jit_done (MonoProfiler *prof, MonoMethod *method, MonoJitInfo* jinfo); static void thread_start (MonoProfiler *prof, uintptr_t tid); static void thread_end (MonoProfiler *prof, uintptr_t tid); + static MonoReflectionType* typemap_java_to_managed (MonoString *java_type_name); #if defined (DEBUG) void set_debug_env_vars (void); diff --git a/src/monodroid/jni/monodroid-glue.cc b/src/monodroid/jni/monodroid-glue.cc index 79f2fe950e1..294fbb99125 100644 --- a/src/monodroid/jni/monodroid-glue.cc +++ b/src/monodroid/jni/monodroid-glue.cc @@ -68,7 +68,7 @@ #include "mkbundle-api.h" #include "monodroid-glue-internal.hh" #include "globals.hh" -#include "xamarin-app.h" +#include "xamarin-app.hh" #include "timing.hh" #ifndef WINDOWS @@ -345,12 +345,14 @@ inline void MonodroidRuntime::gather_bundled_assemblies (jstring_array_wrapper &runtimeApks, size_t *out_user_assemblies_count) { #if defined(DEBUG) || !defined (ANDROID) - for (size_t i = 0; i < AndroidSystem::MAX_OVERRIDES; ++i) { - const char *p = androidSystem.get_override_dir (i); - if (!utils.directory_exists (p)) - continue; - log_info (LOG_ASSEMBLY, "Loading TypeMaps from %s", p); - embeddedAssemblies.try_load_typemaps_from_directory (p); + if (application_config.instant_run_enabled) { + for (size_t i = 0; i < AndroidSystem::MAX_OVERRIDES; ++i) { + const char *p = androidSystem.get_override_dir (i); + if (!utils.directory_exists (p)) + continue; + log_info (LOG_ASSEMBLY, "Loading TypeMaps from %s", p); + embeddedAssemblies.try_load_typemaps_from_directory (p); + } } #endif @@ -918,6 +920,8 @@ MonodroidRuntime::lookup_bridge_info (MonoDomain *domain, MonoImage *image, cons void MonodroidRuntime::init_android_runtime (MonoDomain *domain, JNIEnv *env, jclass runtimeClass, jobject loader) { + mono_add_internal_call ("Java.Interop.TypeManager::monodroid_typemap_java_to_managed", reinterpret_cast(typemap_java_to_managed)); + struct JnienvInitializeArgs init = {}; init.javaVm = osBridge.get_jvm (); init.env = env; @@ -1400,6 +1404,12 @@ MonodroidRuntime::create_and_initialize_domain (JNIEnv* env, jclass runtimeClass return domain; } +MonoReflectionType* +MonodroidRuntime::typemap_java_to_managed (MonoString *java_type_name) +{ + return embeddedAssemblies.typemap_java_to_managed (java_type_name); +} + inline void MonodroidRuntime::Java_mono_android_Runtime_initInternal (JNIEnv *env, jclass klass, jstring lang, jobjectArray runtimeApksJava, jstring runtimeNativeLibDir, jobjectArray appDirs, jobject loader, diff --git a/src/monodroid/jni/xamarin-app.h b/src/monodroid/jni/xamarin-app.h deleted file mode 100644 index 96812303a4e..00000000000 --- a/src/monodroid/jni/xamarin-app.h +++ /dev/null @@ -1,42 +0,0 @@ -// Dear Emacs, this is a -*- C++ -*- header -#ifndef __XAMARIN_ANDROID_TYPEMAP_H -#define __XAMARIN_ANDROID_TYPEMAP_H - -#include - -#include "monodroid.h" - -struct TypeMapHeader -{ - uint32_t version; - uint32_t entry_count; - uint32_t entry_length; - uint32_t value_offset; -}; - -struct ApplicationConfig -{ - bool uses_mono_llvm; - bool uses_mono_aot; - bool uses_assembly_preload; - bool is_a_bundled_app; - bool broken_exception_transitions; - uint8_t bound_exception_type; - uint32_t package_naming_policy; - uint32_t environment_variable_count; - uint32_t system_property_count; - const char *android_package_name; -}; - -MONO_API TypeMapHeader jm_typemap_header; -MONO_API uint8_t jm_typemap[]; - -MONO_API TypeMapHeader mj_typemap_header; -MONO_API uint8_t mj_typemap[]; - -MONO_API ApplicationConfig application_config; -MONO_API const char* app_environment_variables[]; -MONO_API const char* app_system_properties[]; - -MONO_API const char* mono_aot_mode_name; -#endif // __XAMARIN_ANDROID_TYPEMAP_H diff --git a/src/monodroid/jni/xamarin-app.hh b/src/monodroid/jni/xamarin-app.hh new file mode 100644 index 00000000000..7f71a0cf1b5 --- /dev/null +++ b/src/monodroid/jni/xamarin-app.hh @@ -0,0 +1,86 @@ +// Dear Emacs, this is a -*- C++ -*- header +#ifndef __XAMARIN_ANDROID_TYPEMAP_H +#define __XAMARIN_ANDROID_TYPEMAP_H + +#include + +#include + +#include "monodroid.h" + +static constexpr uint32_t MODULE_MAGIC = 0x4D544158; // 'XATM', little-endian +static constexpr uint32_t MODULE_INDEX_MAGIC = 0x49544158; // 'XATI', little-endian +static constexpr uint8_t MODULE_FORMAT_VERSION = 1; // Keep in sync with the value in src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs + +struct BinaryTypeMapHeader +{ + uint32_t magic; + uint32_t version; + uint8_t module_uuid[16]; + uint32_t entry_count; + uint32_t duplicate_count; + uint32_t java_name_width; + uint32_t assembly_name_length; +}; + +struct TypeMapIndexHeader +{ + uint32_t magic; + uint32_t version; + uint32_t entry_count; + uint32_t module_file_name_width; +}; + +struct TypeMapModuleEntry +{ + int32_t type_token_id; + uint32_t java_map_index; +}; + +struct TypeMapModule +{ + uint8_t module_uuid[16]; + uint32_t entry_count; + uint32_t duplicate_count; + TypeMapModuleEntry *map; + TypeMapModuleEntry *duplicate_map; + char *assembly_name; + MonoImage *image; + uint32_t java_name_width; + uint8_t *java_map; +}; + +struct TypeMapJava +{ + uint32_t module_index; + int32_t type_token_id; + uint8_t java_name[]; +}; + +struct ApplicationConfig +{ + bool uses_mono_llvm; + bool uses_mono_aot; + bool uses_assembly_preload; + bool is_a_bundled_app; + bool broken_exception_transitions; + bool instant_run_enabled; + uint8_t bound_exception_type; + uint32_t package_naming_policy; + uint32_t environment_variable_count; + uint32_t system_property_count; + const char *android_package_name; +}; + +MONO_API const uint32_t map_module_count; +MONO_API const uint32_t java_type_count; +MONO_API const uint32_t java_name_width; +MONO_API const TypeMapModule map_modules[]; +MONO_API const TypeMapJava map_java[]; + +MONO_API ApplicationConfig application_config; +MONO_API const char* app_environment_variables[]; +MONO_API const char* app_system_properties[]; + +MONO_API const char* mono_aot_mode_name; +#endif // __XAMARIN_ANDROID_TYPEMAP_H diff --git a/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs b/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs index e62c706518b..6dea9d2dda6 100644 --- a/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs +++ b/tests/MSBuildDeviceIntegration/Tests/InstantRunTest.cs @@ -285,15 +285,14 @@ public void InstantRunFastDevTypemaps ([Values ("dx", "d8")] string dexTool) Assert.IsTrue (b.Install (proj), "packaging should have succeeded. 0"); var apk = Path.Combine (Root, b.ProjectDirectory, proj.IntermediateOutputPath, "android", "bin", "UnnamedProject.UnnamedProject.apk"); - Assert.IsNull (ZipHelper.ReadFileFromZip (apk, "typemap.jm"), $"typemap.jm should NOT be in {apk}."); - Assert.IsNull (ZipHelper.ReadFileFromZip (apk, "typemap.mj"), $"typemap.mj should NOT be in {apk}."); + Assert.IsNull (ZipHelper.ReadFileFromZip (apk, "MonoAndroid.0.typemap"), $"MonoAndroid.0.typemap should NOT be in {apk}."); var logLines = b.LastBuildOutput; Assert.IsTrue (logLines.Any (l => l.Contains ("Building target \"_BuildApkFastDev\" completely.") || l.Contains ("Target _BuildApkFastDev needs to be built")), "Apk should have been built"); Assert.IsTrue (logLines.Any (l => l.Contains ("Building target \"_Upload\" completely")), "_Upload target should have run"); - Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("typemap.jm")), "typemap.jm should have been uploaded"); - Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("typemap.mj")), "typemap.mj should have been uploaded"); + Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("Mono.Android.0.typemap")), "Mono.Android.0.typemap should have been uploaded"); + Assert.IsTrue (logLines.Any (l => l.Contains ("NotifySync CopyFile") && l.Contains ("typemap.index")), "typemap.index should have been uploaded"); } } diff --git a/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj b/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj index 793fa8b7d00..f087d605d4e 100644 --- a/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj +++ b/tests/Runtime-AppBundle/Mono.Android-TestsAppBundle.csproj @@ -21,6 +21,8 @@ d8 aab <_MonoAndroidTest>..\..\src\Mono.Android\Test\ + true + ..\..\product.snk diff --git a/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj b/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj index 39810986789..a8306f81f2e 100644 --- a/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj +++ b/tests/Runtime-MultiDex/Mono.Android-TestsMultiDex.csproj @@ -20,6 +20,8 @@ False true d8 + true + ..\..\product.snk diff --git a/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt b/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt index 429f2b065cc..0ebab922566 100644 --- a/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt +++ b/tests/locales/Xamarin.Android.Locale-Tests/Environment.txt @@ -1,4 +1,9 @@ +debug.mono.debug=1 +debug.mono.log=default,timing=bare +MONO_LOG_LEVEL=debug +MONO_LOG_MASK=asm +MONO_XDEBUG=1 + # Force runtime to use fr-FR locale instead of default locale. LANG=fr-FR __XA_USE_JAVA_DEFAULT_TIMEZONE_ID__=true - diff --git a/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv b/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv index 1bc3ab89ab5..4f19d360e8f 100644 --- a/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv +++ b/tests/msbuild-times-reference/MSBuildDeviceIntegration.csv @@ -3,11 +3,11 @@ Test Name,Time in ms (int) # Data Build_No_Changes,3350 -Build_CSharp_Change,4100 +Build_CSharp_Change,4500 Build_AndroidResource_Change,4250 -Build_AndroidManifest_Change,4250 +Build_AndroidManifest_Change,4500 Build_Designer_Change,3750 -Build_JLO_Change,8150 +Build_JLO_Change,8700 Build_CSProj_Change,9800 Build_XAML_Change,9600 Build_XAML_Change_RefAssembly,6350