Skip to content

Commit

Permalink
Reduce the size of __LINKEDIT Export Info section in stripped binar…
Browse files Browse the repository at this point in the history
…ies (#18408)

# Description

This PR reduces the application's SOD (size on disk) by making
`__LINKEDIT Export Info` section smaller in the stripped Mach-O
binaries.

The feature is controlled by `_ExportSymbolsExplicitly` MSBuild property
and can be disabled by specifying: `-p:_ExportSymbolsExplicitly=true`

Fixes #18332 

# Initial problem

It has been noticed that during stripping, the strip tool does not
resize the export info section after it removes the symbols. Instead it
only zeroes out the entries (achieved by calling `prune_trie` function):

- https://github.com/apple-oss-distributions/cctools/blob/cctools-986/misc/strip.c
- https://github.com/apple-oss-distributions/ld64/blob/ld64-711/src/other/PruneTrie.cpp

Thanks @lambdageek for helping to track this down.

# Approach

As Xamarin build process already collects all the [required symbols][1] needed
for the application to run and preserves them during the strip phase, we can
use the same file to instruct clang toolchain to export only those symbols via
the command line options: `-exported_symbols_list <file>` ([source][2]).

This will make the export info section only include what is necessary for the
runtime - and at the same time eliminate the problem of the `strip` tool which
does not resize stripped symbols.

# Investigation setup

The issue is observable by building and inspecting the test application:
https://github.com/xamarin/xamarin-macios/blob/main/tests/dotnet/MySingleView/MySingleView.csproj
and targeting iOS platform in Release mode.

## Results:

| Measure      | MySingleView - main | MySingleView - this PR | Diff (%) | 
| :---         |                ---: |                   ---: |     ---: |
| SOD (bytes)  |            13668940 |               13458476 |    -1.5% |
| .ipa (bytes) |             4829368 |                4827928 |   -0.03% |

Even though zeroes are compressed well, the SOD is still affected and
unused section takes around 1.5% of the most simplistic app size.
Much bigger impact has been noted when trying out a MAUI iOS template
app with NativeAOT where the `__LINKEDIT Export Info` zeroes take up to
20MB of the SOD, but also with the regular macOS applications:
dotnet/runtime#86707

### Repro current state of MySingleView.app with stripped binary

1. Build the app (you can ignore the need to run the sample, I just did it to
   make sure the changes do not break anything)

```bash
make run-device
``` 

2. Print the load commands - [load_cmds_strip.list][3]

```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_strip.list
```

- We are interested in the export info section:

```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942960
export_size 207712
```

3. Create a hex dump of the export info section - [hex_dump_strip.list][4]

``` bash
xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_strip.list
```

- NOTE: Notice around ~200kb of zeroes from ~0x005ab490 to ~0x005dda00
    
4. Verify exported symbols are correct - [dyld_info_strip.list][5]

``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_strip.list
```

### Repro current state of MySingleView.app with unstripped binary

1. Build the app (the make target preserves the symbols)

```bash
make run-device-no-strip
``` 

2. Print the load commands - [load_cmds_nostrip.list][6]

```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_nostrip.list
```

- We are interested in the export info section:

```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942960
export_size 207712
```

3. Create a hex dump of the export info section - [hex_dump_nostrip.list][7]

``` bash
xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_nostrip.list
```

- Notice that the range: ~ 0x005ab490 to ~ 0x005dda00 now includes exported symbol entries
    
4. Verify exported symbols are correct - [dyld_info_nostrip.list][8]

``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_nostrip.list
```

### Repro the new approach 

1. Build the app (the make target uses the new approach)

```bash
make run-device-export-syms
``` 

2. Print the load commands - [load_cmds_export.list][9]

```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_export.list
```

- We are interested in the export info section ***notice the reduced size of the section***:

```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942432
export_size 1048
```

3. Create a hex dump of the export info section - [hex_dump_export.list][10]

``` bash
xxd -s 5942432 -l 1048 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_export.list
```
    
4. Verify exported symbols are correct - [dyld_info_export.list][11]

``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_export.list
```

---

## Additional benefits

With this approach we could also switch the way strip tool is invoked to
always strip all debug and local symbols via `strip -S -x` instead of passing
the file with symbols to preserve. This would remove the warning that we are
currently getting (which is being ignored):

```
/Applications/Xcode_14.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: warning: removing global symbols from a final linked no longer supported.  Use -exported_symbols_list at link time when building...
```

## Other references:

- https://github.com/qyang-nj/llios/blob/main/exported_symbol/README.md

[1]: https://github.com/xamarin/xamarin-macios/blob/11e7883da04d80c59e4ffbbc955a3e0e0060ff90/tools/dotnet-linker/Steps/GenerateReferencesStep.cs#L38-L44
[2]: https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html
[3]: https://gist.github.com/ivanpovazan/d53f8d10be5e4ea9f39a41ea540aa7fa
[4]: https://gist.github.com/ivanpovazan/60637422f3ff8cb5f437ddd06a21d9c1
[5]: https://gist.github.com/ivanpovazan/352595ad15c2ac02f38dcb3bd4130642
[6]: https://gist.github.com/ivanpovazan/bf700161f2f3691d1d7381c98d4fa0be
[7]: https://gist.github.com/ivanpovazan/44269e4fff5ebd58a4d181451e5c106f
[8]: https://gist.github.com/ivanpovazan/38c5afe076502d514a77420af0e10b01
[9]: https://gist.github.com/ivanpovazan/3f663c3c630005f5a578605d48ba807e
[10]: https://gist.github.com/ivanpovazan/0bb84f64281d05ab20438aeaed64f13c
[11]: https://gist.github.com/ivanpovazan/78b3ba2288f53a2316b9bc46964e7e4f

---------

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
  • Loading branch information
ivanpovazan and rolfbjarne authored Jun 12, 2023
1 parent 707a2db commit e308887
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 4 deletions.
3 changes: 3 additions & 0 deletions dotnet/targets/Xamarin.Shared.Sdk.props
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@

<!-- Setting _RequiresILLinkPack to true so that the ILLink pack is always restored. -->
<_RequiresILLinkPack Condition="'$(_RequiresILLinkPack)' == ''">true</_RequiresILLinkPack>

<!-- Explicitly export symbols using clang command-line option "-exported_symbols_list export_list" -->
<_ExportSymbolsExplicitly Condition="'$(_ExportSymbolsExplicitly)' == ''">true</_ExportSymbolsExplicitly>
</PropertyGroup>

<!-- Set the default RuntimeIdentifier if not already specified. -->
Expand Down
20 changes: 16 additions & 4 deletions dotnet/targets/Xamarin.Shared.Sdk.targets
Original file line number Diff line number Diff line change
Expand Up @@ -1180,6 +1180,10 @@
</PropertyGroup>

<Target Name="_ComputeLinkNativeExecutableInputs" Condition="'$(IsMacEnabled)' == 'true'">
<PropertyGroup>
<_ExportedSymbolsFile Condition="'$(_ExportedSymbolsFile)' == '' and '$(_MtouchSymbolsList)' == ''">/dev/null</_ExportedSymbolsFile> <!-- nothing to export -->
<_ExportedSymbolsFile Condition="'$(_ExportedSymbolsFile)' == ''">$(_MtouchSymbolsList)</_ExportedSymbolsFile>
</PropertyGroup>
<ItemGroup>
<_XamarinMainLibraries Include="$(_XamarinNativeLibraryDirectory)/$(_LibXamarinName)" />
<!-- Link with the libraries shipped with the mono runtime pack -->
Expand All @@ -1196,6 +1200,9 @@
<_MainLinkerFlags Include="-liconv" />
<_MainLinkerFlags Include="-lcompression" />

<_MainLinkerFlags Condition="'$(_ExportSymbolsExplicitly)' == 'true'" Include="-exported_symbols_list" />
<_MainLinkerFlags Condition="'$(_ExportSymbolsExplicitly)' == 'true'" Include="$(_ExportedSymbolsFile)" />

<!-- Here we must add all the files that should make us (re-)link the executable -->
<_LinkNativeExecutableInputs Include="@(_NativeExecutableObjectFiles)" />
<_LinkNativeExecutableInputs Include="@(_XamarinMainLibraries)" />
Expand Down Expand Up @@ -1282,13 +1289,21 @@
<_NativeReferences Include="@(_FileNativeReference)" Condition="'%(_FileNativeReference.LinkToExecutable)' != 'false'" />
</ItemGroup>

<!-- remove the `-u` prefix from `_ReferencesLinkerFlags` so we can give the output file to `strip` -->
<WriteLinesToFile SessionId="$(BuildSessionId)" File="$(_MtouchSymbolsList)" Lines="@(_ReferencesLinkerFlags->'%(Identity)'->Substring (2))" Overwrite="true" />

<ItemGroup>
<_AllLinkerFlags Include="@(_AssemblyLinkerFlags);@(_MainLinkerFlags);@(_CustomLinkFlags)" />
<_AllLinkerFlags Condition="'$(_ExportSymbolsExplicitly)' != 'true'" Include="@(_ReferencesLinkerFlags)" />
</ItemGroup>

<LinkNativeCode
SessionId="$(BuildSessionId)"
DylibRPath="$(_DylibRPath)"
EntitlementsInExecutable="$(_CompiledEntitlements)"
FrameworkRPath="$(_EmbeddedFrameworksRPath)"
Frameworks="@(_NativeExecutableFrameworks);@(_BindingLibraryFrameworks)"
LinkerFlags="@(_AssemblyLinkerFlags);@(_ReferencesLinkerFlags);@(_MainLinkerFlags);@(_CustomLinkFlags)"
LinkerFlags="@(_AllLinkerFlags)"
LinkWithLibraries="@(_XamarinMainLibraries);@(_BindingLibraryLinkWith);@(_MainLinkWith)"
MinimumOSVersion="$(_MinimumOSVersion)"
NativeReferences="@(_NativeReferences)"
Expand All @@ -1301,9 +1316,6 @@
TargetFrameworkMoniker="$(_ComputedTargetFrameworkMoniker)"
/>

<!-- remove the `-u` prefix from `_ReferencesLinkerFlags` so we can give the output file to `strip` -->
<WriteLinesToFile SessionId="$(BuildSessionId)" File="$(_MtouchSymbolsList)" Lines="@(_ReferencesLinkerFlags->'%(Identity)'->Substring (2))" Overwrite="true" />

<ItemGroup>
<!-- Copy the executable from the intermediate directory to the .app -->
<ResolvedFileToPublish Include="$(_IntermediateNativeLibraryDir)$(_NativeExecutableName)">
Expand Down

6 comments on commit e308887

@vs-mobiletools-engineering-service2

This comment was marked as outdated.

@vs-mobiletools-engineering-service2

This comment was marked as outdated.

@vs-mobiletools-engineering-service2

This comment was marked as outdated.

@vs-mobiletools-engineering-service2

This comment was marked as outdated.

@vs-mobiletools-engineering-service2

This comment was marked as outdated.

@vs-mobiletools-engineering-service2

This comment was marked as outdated.

Please sign in to comment.