[vm] dartprecompiledruntime should use MAP_JIT

[dcharkes]

Change dartprecompiledruntime to use MAP_JIT so we can change the MacOS entitlements from allow-unsigned-executable-memory to allow-jit.

cc @mkustermann @a-siva

[zzjjob]

mk

[mkustermann]

@dcharkes Could you please explain why our AOT runtime needs allow-jit? Is it due to using the custom elf loader in certain situations?

[dcharkes]

The experimentation went through running the actual release SDK process (one needs a signing key).

The experiment data from cl/293611898:

(without allow-unsigned-executable-memory)
$ ~/Downloads/dart-sdk/bin/dartaotruntime bla
Killed: 9

(previous sdk, with allow-unsigned-executable-memory)
$ ~/Downloads/dart-sdk-271/bin/dartaotruntime bla
unittest-suite-wait-for-done
unittest-suite-success

The previous discussion is here: #38314 (comment)

I did not explore what exactly was causing the failure. Filing this issue came from the experiment and the documentation of the allow-jit: https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-jit.

[mkustermann]

I did not explore what exactly was causing the failure.

Though that is the interesting part we need to know. Our AOT runtime has generally speaking no need for making pages executable itself.

From #38314 I can remember that there was an issue for signed dart executable to dlopen() an unsigned library. Does enabling that require allow-jit?

[dcharkes]

No, for dlopen we required com.apple.security.cs.disable-library-validation. See cl/294392552.

Though that is the interesting part we need to know. Our AOT runtime has generally speaking no need for making pages executable itself.

Yes, indeed. Unfortunately I do/did not have access to a signing key to experiment myself.

We could possibly download an old SDK which does not have the right entitlements and try if we can catch the Killed: 9 in lldb. (Though without symbols it might be hard to figure out what is going on.) @athomas how easy would it be to create a signed SDK in debug mode with the allow-unsigned-executable-memory entitlement removed from dartprecompiledruntime?

[athomas]

Would a release build with the different entitlements help at all? That'd be easier to do.

[dcharkes]

Yes, having a release dartprecompiledruntime with allow-jit instead of allow-unsigned-executable-memory might help us find what is causing the problem. If we can get a debugger on the place where it fails (instead of the OS just fully killing it) and if the release build has enough symbols to tell us what is the place where it fails.

@mkustermann How important do you feel it is that we spend time on figuring this out?

[athomas]

I provided Daco with a 2.10.0-1.0.dev build signed with allow-jit instead of allow-unsigned-executable-memory.

[dcharkes]

Initial try. Indeed we get killed 9 with those entitlements.

dacoharkes-macbookpro:bin dacoharkes$ pwd
/Users/dacoharkes/Desktop/dart-sdk-allow-jit/bin
dacoharkes-macbookpro:bin dacoharkes$ cp ~/dart-sdk/sdk/xcodebuild/DebugX64/libffi_test_functions.dylib libffi_test_functions.dylib
dacoharkes-macbookpro:bin dacoharkes$ ./dart ~/dart-sdk/sdk/tests/ffi_2/function_test.dart
[works]

dacoharkes-macbookpro:bin dacoharkes$ ./dart2native ~/dart-sdk/sdk/tests/ffi_2/function_test.dart -k aot
Generated: /Users/dacoharkes/dart-sdk/sdk/tests/ffi_2/function_test.aot
dacoharkes-macbookpro:bin dacoharkes$ cp /Users/dacoharkes/dart-sdk/sdk/tests/ffi_2/function_test.aot function_test.aot
dacoharkes-macbookpro:bin dacoharkes$ ./dartaotruntime function_test.aot
Killed: 9

lldb is not happy with signed executables (neither with sudo):

dacoharkes-macbookpro:bin dacoharkes$ pwd
/Users/dacoharkes/Desktop/dart-sdk-allow-jit/bin
dacoharkes-macbookpro:bin dacoharkes$ lldb -- ~/dart-sdk/sdk/xcodebuild/DebugX64/dart ~/dart-sdk/sdk/tests/ffi_2/function_test.dart 
[works]

dacoharkes-macbookpro:bin dacoharkes$ lldb -- ./dart ~/dart-sdk/sdk/tests/ffi_2/function_test.dart 
(lldb) target create "./dart"
Current executable set to './dart' (x86_64).
(lldb) settings set -- target.run-args  "/Users/dacoharkes/dart-sdk/sdk/tests/ffi_2/function_test.dart"
(lldb) r
error: process exited with status -1 (Error 1)

dacoharkes-macbookpro:bin dacoharkes$ lldb -- ./dartaotruntime function_test.aot
(lldb) target create "./dartaotruntime"
Current executable set to './dartaotruntime' (x86_64).
(lldb) settings set -- target.run-args  "function_test.aot"
(lldb) r
error: process exited with status -1 (Error 1)

The issue is not FFI related or opening of dylibs related. Hello world has the same issue:

dacoharkes-macbookpro:bin dacoharkes$ ./dart hello_world.dart 
Hello, World!
dacoharkes-macbookpro:bin dacoharkes$ ./dart2native hello_world.dart -k aot
Generated: /Users/dacoharkes/Desktop/dart-sdk-allow-jit/bin/hello_world.aot
dacoharkes-macbookpro:bin dacoharkes$ ./dartaotruntime hello_world.aot 
Killed: 9

I can dive deeper into this, but it will take time.