[net8.0] Merge main into net8.0. #18436
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…age back to Windows. Fixes xamarin#18402. (xamarin#18419) This fixes a build problem on windows when a project has a project reference to a binding project. The binding resource package from the binding project would lack the manifest, and thus the main project wouldn't detect it as a binding resource package, effectively not seeing any native resource from the binding project. Fixes xamarin#18402.
…ies (xamarin#18408) # Description This PR reduces the application's SOD (size on disk) by making `__LINKEDIT Export Info` section smaller in the stripped Mach-O binaries. The feature is controlled by `_ExportSymbolsExplicitly` MSBuild property and can be disabled by specifying: `-p:_ExportSymbolsExplicitly=true` Fixes xamarin#18332 # Initial problem It has been noticed that during stripping, the strip tool does not resize the export info section after it removes the symbols. Instead it only zeroes out the entries (achieved by calling `prune_trie` function): - https://github.com/apple-oss-distributions/cctools/blob/cctools-986/misc/strip.c - https://github.com/apple-oss-distributions/ld64/blob/ld64-711/src/other/PruneTrie.cpp Thanks @lambdageek for helping to track this down. # Approach As Xamarin build process already collects all the [required symbols][1] needed for the application to run and preserves them during the strip phase, we can use the same file to instruct clang toolchain to export only those symbols via the command line options: `-exported_symbols_list <file>` ([source][2]). This will make the export info section only include what is necessary for the runtime - and at the same time eliminate the problem of the `strip` tool which does not resize stripped symbols. # Investigation setup The issue is observable by building and inspecting the test application: https://github.com/xamarin/xamarin-macios/blob/main/tests/dotnet/MySingleView/MySingleView.csproj and targeting iOS platform in Release mode. ## Results: | Measure | MySingleView - main | MySingleView - this PR | Diff (%) | | :--- | ---: | ---: | ---: | | SOD (bytes) | 13668940 | 13458476 | -1.5% | | .ipa (bytes) | 4829368 | 4827928 | -0.03% | Even though zeroes are compressed well, the SOD is still affected and unused section takes around 1.5% of the most simplistic app size. Much bigger impact has been noted when trying out a MAUI iOS template app with NativeAOT where the `__LINKEDIT Export Info` zeroes take up to 20MB of the SOD, but also with the regular macOS applications: dotnet/runtime#86707 ### Repro current state of MySingleView.app with stripped binary 1. Build the app (you can ignore the need to run the sample, I just did it to make sure the changes do not break anything) ```bash make run-device ``` 2. Print the load commands - [load_cmds_strip.list][3] ```bash otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_strip.list ``` - We are interested in the export info section: ``` cmd LC_DYLD_INFO_ONLY ... export_off 5942960 export_size 207712 ``` 3. Create a hex dump of the export info section - [hex_dump_strip.list][4] ``` bash xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_strip.list ``` - NOTE: Notice around ~200kb of zeroes from ~0x005ab490 to ~0x005dda00 4. Verify exported symbols are correct - [dyld_info_strip.list][5] ``` bash dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_strip.list ``` ### Repro current state of MySingleView.app with unstripped binary 1. Build the app (the make target preserves the symbols) ```bash make run-device-no-strip ``` 2. Print the load commands - [load_cmds_nostrip.list][6] ```bash otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_nostrip.list ``` - We are interested in the export info section: ``` cmd LC_DYLD_INFO_ONLY ... export_off 5942960 export_size 207712 ``` 3. Create a hex dump of the export info section - [hex_dump_nostrip.list][7] ``` bash xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_nostrip.list ``` - Notice that the range: ~ 0x005ab490 to ~ 0x005dda00 now includes exported symbol entries 4. Verify exported symbols are correct - [dyld_info_nostrip.list][8] ``` bash dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_nostrip.list ``` ### Repro the new approach 1. Build the app (the make target uses the new approach) ```bash make run-device-export-syms ``` 2. Print the load commands - [load_cmds_export.list][9] ```bash otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_export.list ``` - We are interested in the export info section ***notice the reduced size of the section***: ``` cmd LC_DYLD_INFO_ONLY ... export_off 5942432 export_size 1048 ``` 3. Create a hex dump of the export info section - [hex_dump_export.list][10] ``` bash xxd -s 5942432 -l 1048 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_export.list ``` 4. Verify exported symbols are correct - [dyld_info_export.list][11] ``` bash dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_export.list ``` --- ## Additional benefits With this approach we could also switch the way strip tool is invoked to always strip all debug and local symbols via `strip -S -x` instead of passing the file with symbols to preserve. This would remove the warning that we are currently getting (which is being ignored): ``` /Applications/Xcode_14.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: warning: removing global symbols from a final linked no longer supported. Use -exported_symbols_list at link time when building... ``` ## Other references: - https://github.com/qyang-nj/llios/blob/main/exported_symbol/README.md [1]: https://github.com/xamarin/xamarin-macios/blob/11e7883da04d80c59e4ffbbc955a3e0e0060ff90/tools/dotnet-linker/Steps/GenerateReferencesStep.cs#L38-L44 [2]: https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html [3]: https://gist.github.com/ivanpovazan/d53f8d10be5e4ea9f39a41ea540aa7fa [4]: https://gist.github.com/ivanpovazan/60637422f3ff8cb5f437ddd06a21d9c1 [5]: https://gist.github.com/ivanpovazan/352595ad15c2ac02f38dcb3bd4130642 [6]: https://gist.github.com/ivanpovazan/bf700161f2f3691d1d7381c98d4fa0be [7]: https://gist.github.com/ivanpovazan/44269e4fff5ebd58a4d181451e5c106f [8]: https://gist.github.com/ivanpovazan/38c5afe076502d514a77420af0e10b01 [9]: https://gist.github.com/ivanpovazan/3f663c3c630005f5a578605d48ba807e [10]: https://gist.github.com/ivanpovazan/0bb84f64281d05ab20438aeaed64f13c [11]: https://gist.github.com/ivanpovazan/78b3ba2288f53a2316b9bc46964e7e4f --------- Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
…commit changes. (xamarin#18426) This fixes a frequent issue where building locally again would create test apps where the registrar would fail due to mismatched runtime versions.
❌ [CI Build] Windows Integration Tests failed ❌❌ Failed ❌ Pipeline on Agent |
📚 [PR Build] Artifacts 📚Packages generatedView packagesPipeline on Agent |
❌ [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) failed ❌Failed tests are:
Pipeline on Agent |
✅ API diff for current PR / commitLegacy Xamarin (No breaking changes)
NET (empty diffs)
✅ API diff vs stableLegacy Xamarin (No breaking changes).NET (No breaking changes)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
❌ [PR Build] Tests on macOS M1 - Mac Ventura (13.0) failed ❌Failed tests are:
Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
🚀 [CI Build] Test results 🚀Test results✅ All tests passed on VSTS: simulator tests. 🎉 All 229 tests passed 🎉 Tests counts✅ bcl: All 69 tests passed. Html Report (VSDrops) Download Pipeline on Agent |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.