Running .NET 7 single file binary on Apple M1 chip causes app to be "killed" #79267
Comments
|
Tagging subscribers to this area: @agocke, @vitek-karas, @VSadov Issue DetailsDiscussed in #79229Originally posted by rcdailey December 5, 2022 dotnet publish MyProject.csproj `
--output publish `
--configuration Release`
--runtime osx-arm64 `
--self-contained true `
-p:PublishSingleFile=true `
-p:IncludeNativeLibrariesForSelfExtract=true `
-p:PublishReadyToRunComposite=true `
-p:PublishReadyToRunShowWarnings=true `
-p:EnableCompressionInSingleFile=true This is the platform I'm running on: My application is not signed, but Mac lets me bypass that for testing purposes. Eventually it just fails like so: I get no further diagnostics. What steps can I take to further diagnose the issue? Why is
|
|
Comment by @am11 : this seems related to #49855. $ dotnet8 publish -o publish -p:PublishSingleFile=true -p:EnableCompressionInSingleFile=true
$ ./publish/helloworldapp # bus error
$ dotnet8 publish -o publish -p:PublishSingleFile=true -p:EnableCompressionInSingleFile=false
$ ./publish/helloworldapp # works |
|
/cc @elinor-fung |
|
It is failing in (lldb) r
Process 95834 launched: '/Users/am11/projects/helloworldapp/publish/helloworldapp' (arm64)
Process 95834 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x1053bdbd4)
frame #0: 0x0000000100576838 helloworldapp`PEImageLayout::ApplyBaseRelocations(bool) + 488
helloworldapp`PEImageLayout::ApplyBaseRelocations:
-> 0x100576838 <+488>: str x10, [x9], #0x8
0x10057683c <+492>: cmp x20, x9
0x100576840 <+496>: csel x20, x20, x9, hi
0x100576844 <+500>: b 0x10057680c ; <+444>
Target 0: (helloworldapp) stopped.
(lldb) clrstack -f
OS Thread Id: 0x127edfa (1)
Child SP IP Call Site
000000016FDFE080 0000000100576838 helloworldapp!PEImageLayout::ApplyBaseRelocations(bool) + 488
000000016FDFE110 0000000100576D74 helloworldapp!ConvertedImageLayout::ConvertedImageLayout(FlatImageLayout*) + 364
000000016FDFE130 0000000100576380 helloworldapp!PEImageLayout::LoadConverted(PEImage*) + 188
000000016FDFE160 00000001000F7BF0 helloworldapp!PEImage::GetOrCreateLayoutInternal(unsigned int) + 68
000000016FDFE190 00000001000F6BCC helloworldapp!PEImage::GetOrCreateLayout(unsigned int) + 160
000000016FDFE1D0 00000001000F4854 helloworldapp!PEAssembly::EnsureLoaded() + 56
000000016FDFE430 000000010005B88C helloworldapp!Assembly::Init(AllocMemTracker*, LoaderAllocator*) + 112
000000016FDFE470 000000010005C650 helloworldapp!Assembly::Create(BaseDomain*, PEAssembly*, DebuggerAssemblyControlFlags, int, AllocMemTracker*, LoaderAllocator*) + 412
000000016FDFE510 000000010008E9C4 helloworldapp!DomainAssembly::Allocate() + 76
000000016FDFE740 000000010008E850 helloworldapp!DomainAssembly::DoIncrementalLoad(FileLoadLevel) + 144
000000016FDFE770 000000010005344C helloworldapp!AppDomain::TryIncrementalLoad(DomainAssembly*, FileLoadLevel, Wrapper<FileLoadLock*, &(void DoNothing<FileLoadLock*>(FileLoadLock*)), &(FileLoadLock::HolderLeave(FileLoadLock*)), 0ul, &(int CompareDefault<FileLoadLock*>(FileLoadLock*, FileLoadLock*)), true>&) + 212
000000016FDFE820 0000000100052908 helloworldapp!AppDomain::LoadDomainAssembly(FileLoadLock*, FileLoadLevel) + 256
000000016FDFE8B0 0000000100052CC4 helloworldapp!AppDomain::LoadDomainAssemblyInternal(AssemblySpec*, PEAssembly*, FileLoadLevel) + 524
000000016FDFE970 000000010004F3AC helloworldapp!SystemDomain::LoadBaseSystemClasses() + 52
000000016FDFE9A0 000000010004F0B0 helloworldapp!SystemDomain::Init() + 580
000000016FDFEA10 000000010057487C helloworldapp!EEStartupHelper() + 1732
000000016FDFEAD0 0000000100573F40 helloworldapp!EEStartup() + 220
000000016FDFEB30 0000000100573E4C helloworldapp!EnsureEEStarted() + 340
000000016FDFEB60 0000000100086728 helloworldapp!CorHost2::Start() + 116
000000016FDFEB90 00000001005783B4 helloworldapp!coreclr_initialize + 724
000000016FDFEC50 000000010001F82C helloworldapp!coreclr_t::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*, char const*, coreclr_property_bag_t const&, std::__1::unique_ptr<coreclr_t, std::__1::default_delete<coreclr_t> >&) + 420
000000016FDFED60 000000010002C654 helloworldapp!(anonymous namespace)::create_coreclr() + 432
000000016FDFEDD0 000000010002C128 helloworldapp!corehost_main + 160
000000016FDFEF20 000000010000C8F4 helloworldapp!fx_muxer_t::handle_exec_host_command(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, host_startup_info_t const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::unordered_map<known_options, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >, known_options_hash, std::__1::equal_to<known_options>, std::__1::allocator<std::__1::pair<known_options const, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > > > const&, int, char const**, int, host_mode_t, bool, char*, int, int*) + 1328
000000016FDFF080 000000010000B9E8 helloworldapp!fx_muxer_t::execute(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int, char const**, host_startup_info_t const&, char*, int, int*) + 860
000000016FDFF190 000000010000848C helloworldapp!hostfxr_main_bundle_startupinfo + 196
000000016FDFF240 000000010004C93C helloworldapp!exe_start(int, char const**) + 1132
000000016FDFF370 000000010004CC20 helloworldapp!main + 152
000000016FDFF3D0 0000000196033E50 dyld!start + 2544
(lldb) disas
helloworldapp`PEImageLayout::ApplyBaseRelocations:
0x100576650 <+0>: sub sp, sp, #0x90
0x100576654 <+4>: stp x28, x27, [sp, #0x30]
0x100576658 <+8>: stp x26, x25, [sp, #0x40]
0x10057665c <+12>: stp x24, x23, [sp, #0x50]
0x100576660 <+16>: stp x22, x21, [sp, #0x60]
0x100576664 <+20>: stp x20, x19, [sp, #0x70]
0x100576668 <+24>: stp x29, x30, [sp, #0x80]
0x10057666c <+28>: add x29, sp, #0x80
0x100576670 <+32>: mov x19, x0
0x100576674 <+36>: ldr x9, [x19, #0x8]!
0x100576678 <+40>: ldr w8, [x19, #0xc]
0x10057667c <+44>: orr w8, w8, #0x4
0x100576680 <+48>: str w8, [x19, #0xc]
0x100576684 <+52>: ldrsw x8, [x9, #0x3c]
0x100576688 <+56>: add x8, x9, x8
0x10057668c <+60>: ldrh w10, [x8, #0x18]
0x100576690 <+64>: cmp w10, #0x10b
0x100576694 <+68>: b.ne 0x1005766a8 ; <+88>
0x100576698 <+72>: ldr w11, [x8, #0x34]
0x10057669c <+76>: subs x25, x9, x11
0x1005766a0 <+80>: b.ne 0x1005766b4 ; <+100>
0x1005766a4 <+84>: b 0x1005768f4 ; <+676>
0x1005766a8 <+88>: ldr x11, [x8, #0x30]
0x1005766ac <+92>: subs x25, x9, x11
0x1005766b0 <+96>: b.eq 0x1005768f4 ; <+676>
0x1005766b4 <+100>: cmp w10, #0x10b
0x1005766b8 <+104>: mov w9, #0x88
0x1005766bc <+108>: mov w10, #0x78
0x1005766c0 <+112>: csel x9, x10, x9, eq
0x1005766c4 <+116>: add x8, x8, x9
0x1005766c8 <+120>: ldp w1, w28, [x8, #0x28]
0x1005766cc <+124>: mov x0, x19
0x1005766d0 <+128>: mov w2, #0x0
0x1005766d4 <+132>: bl 0x100310e94 ; PEDecoder::GetRvaData(unsigned int, IsNullOK) const
0x1005766d8 <+136>: cbz w28, 0x100576914 ; <+708>
0x1005766dc <+140>: mov x12, #0x0
0x1005766e0 <+144>: mov x13, #0x0
0x1005766e4 <+148>: mov w24, #0x0
0x1005766e8 <+152>: mov x11, #0x0
0x1005766ec <+156>: stp x0, xzr, [sp, #0x20]
0x1005766f0 <+160>: mov w27, #0x0
0x1005766f4 <+164>: mov x23, #0x1fffffff8
0x1005766f8 <+168>: str w28, [sp, #0x1c]
0x1005766fc <+172>: b 0x100576718 ; <+200>
0x100576700 <+176>: mov x11, x21
0x100576704 <+180>: sub x8, x20, x11
0x100576708 <+184>: str x8, [sp, #0x28]
0x10057670c <+188>: add w27, w22, w27
0x100576710 <+192>: cmp w27, w28
0x100576714 <+196>: b.hs 0x1005768a8 ; <+600>
0x100576718 <+200>: add x8, x0, w27, uxtw
0x10057671c <+204>: add x26, x8, #0x8
0x100576720 <+208>: ldp w20, w22, [x8]
0x100576724 <+212>: ldr x8, [x19]
0x100576728 <+216>: add x21, x8, x20
0x10057672c <+220>: sub x8, x21, x12
0x100576730 <+224>: cmp x8, x13
0x100576734 <+228>: b.lo 0x1005767f4 ; <+420>
0x100576738 <+232>: mov x28, x11
0x10057673c <+236>: tst w24, #0xf0
0x100576740 <+240>: b.eq 0x10057674c ; <+252>
0x100576744 <+244>: mov w0, #0x0
0x100576748 <+248>: bl 0x1005fd110 ; PAL_JitWriteProtect
0x10057674c <+252>: ldrh w8, [x26]
0x100576750 <+256>: and w8, w8, #0xfff
0x100576754 <+260>: add w1, w20, w8
0x100576758 <+264>: mov x0, x19
0x10057675c <+268>: bl 0x1003106ec ; PEDecoder::RvaToSection(unsigned int) const
0x100576760 <+272>: mov x24, x0
0x100576764 <+276>: ldr w1, [x0, #0xc]
0x100576768 <+280>: mov x0, x19
0x10057676c <+284>: mov w2, #0x0
0x100576770 <+288>: bl 0x100310e94 ; PEDecoder::GetRvaData(unsigned int, IsNullOK) const
0x100576774 <+292>: mov x12, x0
0x100576778 <+296>: ldr w13, [x24, #0x10]
0x10057677c <+300>: ldr w8, [x24, #0x24]
0x100576780 <+304>: tbnz w8, #0x1f, 0x100576798 ; <+328>
0x100576784 <+308>: tbnz w8, #0x1d, 0x1005767a8 ; <+344>
0x100576788 <+312>: mov w8, #0x0
0x10057678c <+316>: ldr x0, [sp, #0x20]
0x100576790 <+320>: mov x11, x28
0x100576794 <+324>: b 0x1005767dc ; <+396>
0x100576798 <+328>: mov w24, #0x0
0x10057679c <+332>: ldr x0, [sp, #0x20]
0x1005767a0 <+336>: mov x11, x28
0x1005767a4 <+340>: b 0x1005767ec ; <+412>
0x1005767a8 <+344>: mov w0, #0x1
0x1005767ac <+348>: mov x20, x12
0x1005767b0 <+352>: mov x23, x19
0x1005767b4 <+356>: mov x19, x13
0x1005767b8 <+360>: bl 0x1005fd110 ; PAL_JitWriteProtect
0x1005767bc <+364>: mov x13, x19
0x1005767c0 <+368>: mov x19, x23
0x1005767c4 <+372>: mov x12, x20
0x1005767c8 <+376>: ldr w9, [x24, #0x24]
0x1005767cc <+380>: and w8, w9, #0x20000000
0x1005767d0 <+384>: ldr x0, [sp, #0x20]
0x1005767d4 <+388>: mov x11, x28
0x1005767d8 <+392>: tbnz w9, #0x1f, 0x100576898 ; <+584>
0x1005767dc <+396>: cmp w8, #0x0
0x1005767e0 <+400>: mov w8, #0x20
0x1005767e4 <+404>: mov w9, #0x2
0x1005767e8 <+408>: csel w24, w9, w8, eq
0x1005767ec <+412>: ldr w28, [sp, #0x1c]
0x1005767f0 <+416>: mov x23, #0x1fffffff8
0x1005767f4 <+420>: add x8, x22, x23
0x1005767f8 <+424>: lsr x9, x8, #1
0x1005767fc <+428>: cbz w9, 0x10057670c ; <+188>
0x100576800 <+432>: mov x20, #0x0
0x100576804 <+436>: ubfx x8, x8, #1, #32
0x100576808 <+440>: b 0x100576818 ; <+456>
0x10057680c <+444>: add x26, x26, #0x2
0x100576810 <+448>: subs x8, x8, #0x1
0x100576814 <+452>: b.eq 0x100576848 ; <+504>
0x100576818 <+456>: ldrh w9, [x26]
0x10057681c <+460>: and w10, w9, #0xf000
0x100576820 <+464>: cmp w10, #0xa, lsl #12 ; =0xa000
0x100576824 <+468>: b.ne 0x10057680c ; <+444>
0x100576828 <+472>: and x9, x9, #0xfff
0x10057682c <+476>: add x9, x21, x9
0x100576830 <+480>: ldr x10, [x9]
0x100576834 <+484>: add x10, x10, x25
-> 0x100576838 <+488>: str x10, [x9], #0x8
0x10057683c <+492>: cmp x20, x9
0x100576840 <+496>: csel x20, x20, x9, hi
0x100576844 <+500>: b 0x10057680c ; <+444>
0x100576848 <+504>: tst w24, #0xf0
0x10057684c <+508>: b.eq 0x10057670c ; <+188>
0x100576850 <+512>: cbz x20, 0x10057670c ; <+188>
0x100576854 <+516>: ldr x8, [sp, #0x28]
0x100576858 <+520>: add x8, x11, x8
0x10057685c <+524>: add x8, x8, #0x1, lsl #12 ; =0x1000
0x100576860 <+528>: cmp x21, x8
0x100576864 <+532>: b.hs 0x100576870 ; <+544>
0x100576868 <+536>: cmp x21, x11
0x10057686c <+540>: b.hs 0x100576704 ; <+180>
0x100576870 <+544>: cbz x11, 0x100576700 ; <+176>
0x100576874 <+548>: mov x26, x11
0x100576878 <+552>: stp x13, x12, [sp, #0x8]
0x10057687c <+556>: bl 0x1006174dc ; GetCurrentProcess
...cc @janvorli |
Note: if we use x64 (pass |
|
Thanks for the quick turnaround on this. Will this be fixed in .net 7 as a hot fix release? Until then, is there a workaround that I can use to get this working? |
|
I should also add, I observed this same behavior almost a year ago on .net 6 as well. |
If flipping |
Having `EnableCompressionInSingleFile` set to `true` causes a crash when launching Recyclarr on Apple M1 (arm64) chips. As a workaround, disable this flag. Eventually this should be re-enabled once the following issue is resolved: dotnet/runtime#79267
@am11 sadly, this did not work. Here is the build I tested: My project is entirely open source so you are welcome to experiment with it. I'm still looking for a workaround! |
|
@VSadov for investigation |
|
I would definitely take this for back port to 7 |
|
Current theory is this is caused by intersection of compression and W^X features. The memory that we allocate for decompression needs to have special attributes (MAP_JIT), so that the protection could be flipped during relocations between writeable and executable. |
@rcdailey, I was able to run that osx-arm64 recycler executable on M1 machine. There are few actions I needed to perform after downloading the zip: # extract the app
$ unzip recyclarr-osx-arm64.zip
# remove apple quarantine attribute
$ xattr -d com.apple.quarantine recyclarr
# set executable bit
$ chmod +x ./recyclarr
# code sign it
$ codesign --force --deep -s - ./recyclarr
# run the app
$ ./recyclarr
recyclarr v3.0.1 (4.Branch.disable-compression.Sha.4ff513870d14a92aad8e68836fb155ddb525cf66)
USAGE
recyclarr [options]
recyclarr [command] [...]
OPTIONS
-h|--help Shows help text.
--version Shows version information.
COMMANDS
create-config Create a starter YAML configuration file
migrate Perform any migration steps that may be needed between versions
radarr Perform operations on a Radarr instance
sonarr Perform operations on a Sonarr instance
You can run `recyclarr [command] --help` to show help on a specific command.It doesn't work when compression is enabled (e.g. if we download the artifacts from the current release https://github.com/recyclarr/recyclarr/releases/tag/v3.0.0 which doesn't have |
|
Thank you for checking @am11. Can you confirm that it does not work if you run without signing it and running the other steps? I'm pretty sure the users that reported the crash to me did not run those steps. |
|
Yup, I can confirm that without:
Code-signing is mandatory on recent versions of macOS. Also, I think it's better to publish macOS executable from macOS machine and Linux from Linux machine, and create a |
|
Thank you for those tips, I will make those changes. In order to code sign my executable for MacOS, do I need to pay for an Apple developer subscription? Do I need to request a certificate from Apple in order to do that? I will try googling a bit, but my recollection from years ago is that this kind of thing required paying to be an apple developer. MacOS is not something I use so I'm mostly unfamiliar with their process. |
|
I'm actually not super familiar with Apple's rules on this.
@am11 does that really work? I thought ad-hoc signing only applied to the local machine -- if you tried to copy the binaries anywhere else they would have to be either re-signed or signed with an Apple developer cert. |
|
You are right; codesign step will still be needed on user's machine unless the app is notarized with these entitlements.
Yup, and it looks like a recurring cost; requires annual license renewal. |
|
That is another example of cross-OS publishing issue. I was suggesting to avoid cross-OS publishing altogether; cross-architecture publishing for same OS is fine. On GitHub Actions it is possible by using dedicated jobs: |
|
So there was a time in the past where I did exactly what you're suggesting: A matrix build where on linux I build for linux, mac I build for mac, and windows I build for windows. However, I found that for linux / mac builds, no version information was in the executable unless I build them on Windows for those RIDs. The reason I shared the issue was to show that there are reasons why it's difficult to do it the way you recommend as of current. |
I'm curious about that - AFAIK linux/mac doesn't have a common format to store version information in the executable. On Windows it's stored as a Win32 resource (and due to #3828 we're currently unable to write the Win32 resources on non-Windows machines). But on Linux/mac we've never written the version information anywhere. You will have a PE Win32 version resource in the managed |
|
I sadly don't know the inner workings of the whole thing. What I observed is that GitVersion / Nerdbank.GitVersioning libraries were not able to pull version information from the executable for my console application. That was a long time ago, though. I'm revisiting the suggestion from @am11 to build on each platform so I'll see if I run into the issue again. I apologize for bringing up an unrelated issue. It was not my intention to cause a distraction. |
@am11 Are these instructions the guidance I should follow for signing my OSX application? It seems like a lot of work. I was hoping that |
I haven't seen that link before, so I cannot say if/how that works. The simpler solution I was pointing to was to publish the macOS artifacts from macOS machine and pack it as a tar file (to preserve permission bits) and ask user to run |
|
I wanted to sign the application via my GitHub workflow. I went ahead and purchased an apple developer membership so that I can do that. Seems like I need to do it by hand and dotnet tooling only does adhoc signing. |
|
Yes, the entire signing and notarization process is quite involved. Here is a flat script with raw steps I've pieced together for This is the .app version, you can also have .dmg, .pkg and .zip version to distribute signed software on macOS. This zip has to be created by ditto(1) utility which is usually present in PATH. |
|
Is notarization required in order to distribute my executable to users without causing them trouble? The end goal, in my opinion, is for the application to execute normally as it would on Linux (which doesn't require all of this process). The signing part seems easy; I just want to know if I can stop there or if I should do notarization as well. I do not plan to distribute my application via the Mac/Apple store. |
We don't need any of this process, not even signing, if that's your goal all along. Your app is fine: #79267 (comment) |
|
I don't want my users to have to sign my application, my goal is to make my application ready to un-tarball and run with no extra fuss. Given that requirement, it's not really true that we don't need any of the process, right? If I'm wrong, where am I confused? |
|
You can use the posted script if you want app bundle (.app) model, then you don't need to manually zip or tar anything. You can skip notarization step 4 if you don't want "full trust".
Signing is a partial trust. You don't need to purchase anything for it, just sign up for a free developer certificate (Apple ID). Notarization is full trust; Apple vouches for your app. That has a recurring cost. So if you have purchased something and you don't need notarization, you can revert the transaction and get the reimbursement. |
|
The Developer ID certificate option was not even available on the developer apple website without a membership. Where am I supposed to go to get a "free developer certificate"? The only alternative I was able to find was signing with a self-signed cert, but that would not get rid of the prompts users see when they run my app. Sorry for all the confusion on my part. I'm learning a lot, though, so thank you for this. EDIT: Found some instruction here, but:
|
Sorry, this is a rough edge at the moment. Unfortunately we don't really have any say in the Mac requirements. They are fully controlled by Apple. This is why we don't have any documentation on signing -- Apple is the ultimate authority on how to sign on their platform. The only documentation I have found from them is https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html Similarly, this is why we don't do anything in Lastly, we don't support signing on non-Apple platforms simply because Apple does not support signing on non-Apple platforms. |
|
Thanks for all the support. I know I've taken this issue on quite a detour. At this point I've learned a lot and I will take further questions to the Apple dev forums. Thanks so much for everything and I apologize for going off topic. |
|
Thanks for discussion! I know this is a sore spot, so I'm very interested in your feedback. Once you figure out what you need, I would love to know what, if anything, we could do to make this simpler without making promises that we can't keep (e.g., by supporting code signing on non-Apple platforms). |
|
This is resolved. The fix for Backport to .NET 7: #80283. |
ghost
locked as resolved and limited conversation to collaborators
This issue has been fixed: dotnet/runtime#79267
Discussed in #79229
Originally posted by rcdailey December 5, 2022
I have a Github Workflow that compiles my .NET 7 console application using a windows agent. It runs this in a powershell script:
This is the platform I'm running on:
My application is not signed, but Mac lets me bypass that for testing purposes. Eventually it just fails like so:
I get no further diagnostics. What steps can I take to further diagnose the issue? Why is
osx-arm64not working butosx-x64works just fine?The text was updated successfully, but these errors were encountered: