Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AIO #1

Closed
wants to merge 21 commits into from
Closed

AIO #1

wants to merge 21 commits into from

Conversation

adamsitnik
Copy link
Owner

No description provided.

adamsitnik pushed a commit that referenced this pull request Mar 17, 2020
This adds support in the JIT emitter for Vector Load/Store structure instructions (C3.2.10 - Arm
Architecture Reference Manual):

- LD1 (1-4 registers)
- LD2
- LD3
- LD4
- LD1R
- LD2R
- LD3R
- LD4R
- ST1 (1-4 registers)
- ST2
- ST3
- ST4

in the following addressing modes:

- Base register only
- Post-indexed by a 64-bit register
- Post-indexed by an immediate, equal to the number of bytes transferred

Also adds support in JitDump for printing of

* A SIMD vector register list.
  For example, ld1     {v5.16b, v6.16b, v7.16b, v8.16b}, [x9]

* A SIMD vector element list. 
  For example, st1     {v0.b}[3], [x1],#1

// these fields swap for big endian
// https://github.com/torvalds/linux/blob/0a679e13ea30f85a1aef0669ee0c5a9fd7860b34/include/uapi/linux/aio_abi.h#L77-L83
private uint _swappedField_1;
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a little bit tricky: Linux Kernel has different order of these two fields depending on the endianess...

I've encapsulated that my making them private and adding internal properties with some simple logic that handles the endianess

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't bother too much for fields that aren't used.

@@ -56,6 +56,10 @@
#if HAVE_LINUX_CAN_H
#include <linux/can.h>
#endif
#if HAVE_LINUX_AIO
#include <linux/aio_abi.h>
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important: the AIO is available on Linux, but not on all Unixes (like MacOS), hence I needed this switch

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AIO is Linux only. No other OS will have this. Using #if defined(__linux) is also an option.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we do not support kernels where linux/aio_abi.h is not available, right?

return false;
}

Debug.Assert(oldState == (int)State.Waiting);
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to make sure that we NEVER run into the situation where we set to running a non-waiting operation (Debug.Assert is not preventing this for Release builds)

@@ -310,6 +310,110 @@ public void DoAbort()

public abstract void InvokeCallback(bool allowPooling);

internal virtual bool TryBatch(SocketAsyncContext context, int id, ref Interop.Sys.IoControlBlock ioControlBlock) => false;
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: Using ref to keep it both safe and fast

@@ -310,6 +310,110 @@ public void DoAbort()

public abstract void InvokeCallback(bool allowPooling);

internal virtual bool TryBatch(SocketAsyncContext context, int id, ref Interop.Sys.IoControlBlock ioControlBlock) => false;

internal virtual void HandleBatchEvent(in Interop.Sys.IoEvent ioControlBlock)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: using in to avoid expensive struct copies on Linux

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It only works for readonly structs AFAIK.

return SocketError.AddressNotAvailable;
case Interop.Error.EAFNOSUPPORT:
return SocketError.AddressFamilyNotSupported;
case (Interop.Error)11:
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important: this method requires a rewrite - AIO is Linux system call and it returns different error codes that what we have in Interop.Error. A good example is Interop.Error.EAGAIN which is not 11 (while here it must be 11)

Copy link

@tmds tmds Apr 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would make sense for this to be in the pal IoGetEvents. The errnos returned can be mapped to SocketError in the IoEvent.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tmds very good idea!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamsitnik @tmds my understanding is that none of these errors codes are mapped correctly.
io_event.Res will contain Linux error codes which have nothing to do with the pal_errno.h codes starting with 0x1. I'm planning to map them in IoGetEvents as suggested, reusing the common error mapping logic.
Please correct me if my understanding is wrong!

Note: currently I'm debugging a case of ESPIPE (29 - "Illegal seek").

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please correct me if my understanding is wrong!

@antonfirsov You are correct

void IThreadPoolWorkItem.Execute() => AssociatedContext.ProcessAsyncReadOperation(this);
void IThreadPoolWorkItem.Execute()
{
if (IsCompleted)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for batched operations, we dispatch them to thread pool to more or less only run continuation on thread pool thread

@@ -136,6 +140,8 @@ public bool TryRegister(SafeSocketHandle socket, out Interop.Error error)
//
private bool IsFull { get { return _nextHandle == MaxHandles; } }

private bool IsAio => _aioContext.Ring != null;
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might need a cleaner check or even moving entire AIO logic to a standalone type

}
else
{
Debug.Assert(Interop.Sys.IsAioSupported() == false, "When AIO is supported IoSetup should always succeed");
Copy link
Owner Author

@adamsitnik adamsitnik Apr 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important: IoSetup might fail - on MacOS the API is simply missing, on Linux the user might not have enough of resources for the method to succeed

I added this check to make sure that our Debug tests on Linux test AIO for real

{
shutdown = true;
// there was an error, we use the non-buffered execution path (this should be very rare)
context.HandleEvents(socketEvent.Events);
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important: epoll can return read, write or error. If it's error, we fall back to non-AIO code path

}

// todo: perf: avoid the syscall by using a well known pattern that reads from the ring
result = Interop.Sys.IoGetEvents(_aioContext.Ring, batchedCount, batchedCount, _aioEvents);
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

important: it's possible to avoid this sys call like @tmds does that in https://github.com/tmds/Tmds.LinuxAsync/blob/11233fc8fc80862533e319f4d22855347981aace/src/Tmds.LinuxAsync/EPollAsyncEngine.LinuxAio.cs#L203

it was not hot on the profile, so it was sitting on my todo list

{
throw new InternalException(err);
shutdown = true;
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the shutdown is implemented in a very interesting way: the epoll does not support canceling, but afaik we have some pipe where we write cancel request, which is represented by a file descriptior that is monitored by epool itself. So if there is new data in it, epoll receives a notification and we stop here.

@adamsitnik
Copy link
Owner Author

@antonfirsov I've added all the comments. If it makes it any easier for you, please feel free to squash all the commits and|or rewrite git history (there are some commits of shame here :D )

internal static partial class Sys
{
[StructLayout(LayoutKind.Sequential)]
internal struct IoEvent
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tmds you rather mean the C struct here I guess?

}

[DllImport(Libraries.SystemNative, EntryPoint = "SystemNative_IsAioSupported")]
internal static extern bool IsAioSupported();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be implicitly done based on IoSetup return value.

@@ -362,6 +488,7 @@ private abstract class SendOperation : WriteOperation
private sealed class BufferMemorySendOperation : SendOperation
{
public Memory<byte> Buffer;
internal MemoryHandle PinHandle;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These handles are only used while performing the operation against AIO. They could be in a List<MemoryHandle> that is populated during the use of AIO, instead of per operation.

With a single handle, multi buffer sends/receives can't be performed using AIO.


ioControlBlock.AioData = (ulong)id;
ioControlBlock.AioLioOpcode = Interop.Sys.IoControlBlockFlags.IOCB_CMD_PWRITE;
ioControlBlock.AioFildes = (uint)context._socket.DangerousGetHandle().ToInt32();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The handle reference count isn't incremented.

{
ErrorCode = Map((Interop.Error)(-ioControlBlock.Res));

if (CanRetry(ErrorCode))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to handle EINTR too.
I'm not sure if it can happen, but better be safe than sorry.
In case of EINTR, the operation should be tried again.

_state = QueueState.Processing;
nextOperation = _tail.Next;

while (batchedCount < ioControlBlocks.Length && nextOperation.TryBatch(context, batchedCount, ref ioControlBlocks[batchedCount]))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider batching a single operation here.

ReadOnlySpan<Interop.Sys.IoEvent> events = new ReadOnlySpan<Interop.Sys.IoEvent>(_aioEvents, batchedCount);
for (int i = 0; i < events.Length; i++)
{
batchedOperations[events[i].Data].HandleBatchEvent(in events[i]);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some code path that ensures other operations that are pending in the queue get executed?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tom, I am not sure if I understand. We batch what we can and dispatch the rest, then here we handle only what was batched.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dispatch the rest

We can't dispatch the rest. The operations need to be handled sequentially.

We also need to take care of new operations being added to the queue while the batched operations are executing. So at the end of HandleBatchEvent we must check the queue.

Copy link

@antonfirsov antonfirsov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I resolved the conflicts in my branch, and currently in a phase of trying it out, will come back with more questions later.

internal static partial class Sys
{
[StructLayout(LayoutKind.Sequential)]
internal struct IoEvent

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tmds you rather mean the C struct here I guess?

@@ -56,6 +56,10 @@
#if HAVE_LINUX_CAN_H
#include <linux/can.h>
#endif
#if HAVE_LINUX_AIO
#include <linux/aio_abi.h>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we do not support kernels where linux/aio_abi.h is not available, right?

@@ -310,6 +310,110 @@ public void DoAbort()

public abstract void InvokeCallback(bool allowPooling);

internal virtual bool TryBatch(SocketAsyncContext context, int id, ref Interop.Sys.IoControlBlock ioControlBlock) => false;

internal virtual void HandleBatchEvent(in Interop.Sys.IoEvent ioControlBlock)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It only works for readonly structs AFAIK.

@adamsitnik
Copy link
Owner Author

I guess we do not support kernels where linux/aio_abi.h is not available, right?

The file is in the Unix folder, I assumed that it can be compiled for macOS

@adamsitnik adamsitnik closed this Feb 25, 2021
adamsitnik pushed a commit that referenced this pull request May 24, 2021
…tnet#52769)

Transition to GC Unsafe mode on every MONO_RT_EXTERNAL_ONLY function in
reflection.c

In particular, fix mono_reflection_type_from_name which is used in
https://github.com/xamarin/xamarin-android/blob/681887ebdbd192ce7ce1cd02221d4939599ba762/src/monodroid/jni/embedded-assemblies.cc#L350

Fixes stack traces like

```
05-14 08:06:12.848 31274 31274 F DEBUG   :       #00 pc 00000b99  [vdso] (__kernel_vsyscall+9)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #1 pc 0005ad68  /apex/com.android.runtime/lib/bionic/libc.so (syscall+40) (BuildId: 6e3a0180fa6637b68c0d181c343e6806)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #2 pc 00076511  /apex/com.android.runtime/lib/bionic/libc.so (abort+209) (BuildId: 6e3a0180fa6637b68c0d181c343e6806)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #3 pc 0002afcd  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+141) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
05-14 08:06:12.848 31274 31274 F DEBUG   :       #4 pc 00112c5d  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (eglib_log_adapter+141) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #5 pc 00020fdf  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (monoeg_g_logv+175) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #6 pc 0002113a  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (monoeg_g_log+42) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #7 pc 00128892  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_transition_do_blocking+258) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #8 pc 0012a406  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_enter_gc_safe_region_unbalanced_with_info+134) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #9 pc 0012a27e  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_enter_gc_safe_region_internal+46) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #10 pc 000799a7  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_loader_lock+71) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #11 pc 000447a1  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_create_from_typedef+129) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #12 pc 0003c073  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_get_checked+99) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #13 pc 0003cc0f  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_from_name_checked_aux+735) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #14 pc 00037989  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_from_name_checked+73) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #15 pc 000cc5f4  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_get_type_internal+132) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       #16 pc 000c9bce  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_get_type_with_rootimage+126) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       dotnet#17 pc 000ca204  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (_mono_reflection_get_type_from_info+292) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       dotnet#18 pc 000ca06e  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_type_from_name_checked+334) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       dotnet#19 pc 000c9f01  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_type_from_name+49) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac)
05-14 08:06:12.849 31274 31274 F DEBUG   :       dotnet#20 pc 0001b40b  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(char const*)+427) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
05-14 08:06:12.849 31274 31274 F DEBUG   :       dotnet#21 pc 0001b551  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(_MonoString*)+113) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
05-14 08:06:12.849 31274 31274 F DEBUG   :       dotnet#22 pc 000211a7  /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::typemap_java_to_managed(_MonoString*)+39) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1)
```
adamsitnik pushed a commit that referenced this pull request May 24, 2021
…2915)

* [build] Define NO_UNALIGNED_ACCESS for 32-bit arm platforms

Possibly related to crashes on Android like this:

```
05-18 10:59:07.466 17076 17076 F libc    : Fatal signal 7 (SIGBUS), code 1 (BUS_ADRALN), fault addr 0xb9c95a41 in tid 17076 (simplehellomaui), pid 17076 (simplehellomaui)
05-18 10:59:07.501 17104 17104 I crash_dump32: obtaining output fd from tombstoned, type: kDebuggerdTombstone
05-18 10:59:07.502   989   989 I tombstoned: received crash request for pid 17076
05-18 10:59:07.503 17104 17104 I crash_dump32: performing dump of process 17076 (target tid = 17076)
05-18 10:59:07.512 17104 17104 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
05-18 10:59:07.512 17104 17104 F DEBUG   : Build fingerprint: 'google/crosshatch/crosshatch:11/RQ2A.210405.005/7181113:user/release-keys'
05-18 10:59:07.512 17104 17104 F DEBUG   : Revision: 'MP1.0'
05-18 10:59:07.512 17104 17104 F DEBUG   : ABI: 'arm'
05-18 10:59:07.515 17104 17104 F DEBUG   : Timestamp: 2021-05-18 10:59:07+0200
05-18 10:59:07.515 17104 17104 F DEBUG   : pid: 17076, tid: 17076, name: simplehellomaui  >>> com.microsoft.simplehellomaui <<<
05-18 10:59:07.515 17104 17104 F DEBUG   : uid: 10364
05-18 10:59:07.515 17104 17104 F DEBUG   : signal 7 (SIGBUS), code 1 (BUS_ADRALN), fault addr 0xb9c95a41
05-18 10:59:07.515 17104 17104 F DEBUG   :     r0  bb4a5cd0  r1  b9c95a49  r2  00000000  r3  e94c7520
05-18 10:59:07.515 17104 17104 F DEBUG   :     r4  0000000c  r5  00000000  r6  ff843c50  r7  ff843e70
05-18 10:59:07.515 17104 17104 F DEBUG   :     r8  b69547f8  r9  e99eac50  r10 00000000  r11 00000021
05-18 10:59:07.515 17104 17104 F DEBUG   :     ip  e94c74f0  sp  ff843c48  lr  bb31e0dd  pc  bb3a4d24
05-18 10:59:07.531   709   709 E Layer   : [Surface(name=Task=1)/@0x52e6b1a - animation-leash#0] No local sync point found
05-18 10:59:07.532   709   709 E Layer   : [Surface(name=Task=1571)/@0x9c90165 - animation-leash#0] No local sync point found
05-18 10:59:07.706 17104 17104 F DEBUG   : backtrace:
05-18 10:59:07.707 17104 17104 F DEBUG   :       #00 pc 000ddd24  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (mono_method_to_ir+9232) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #1 pc 000d7777  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (inline_method+622) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #2 pc 000ec0a3  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (mono_method_to_ir+67470) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #3 pc 000cda6d  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (mini_method_compile+2264) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #4 pc 000cf413  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (mono_jit_compile_method_inner+50) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #5 pc 000d1d7f  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (mono_jit_compile_method_with_opt+1766) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #6 pc 0012d94d  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (common_call_trampoline+832) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #7 pc 0012d5cb  /data/app/~~J4DFQ3c1v2YGrEurX7TNjg==/com.microsoft.simplehellomaui-_jGGPiZpZ3yT-QCTNDcgvQ==/lib/arm/libmonosgen-2.0.so (mono_magic_trampoline+62) (BuildId: d0a4e41a500357a621884b64f6ca8533b62a664b)
05-18 10:59:07.707 17104 17104 F DEBUG   :       #8 pc 0000006a <anonymous:b7986000>
```

* move to host/target sections
adamsitnik pushed a commit that referenced this pull request Jul 2, 2021
…et#53792)

I have expanded the PerfMap format produced by Crossgen2 and
R2RDump to produce metadata in form of pseudo-symbol records with
high addresses. In this version I have implemented four metadata
entries - output GUID, target OS, target architecture and perfmap
format version number.  I have verified for System.Private.CoreLib
and for the composite framework that Crossgen2 and R2RDump
produce identical metadata.

To facilitate a smooth transition to the new perfmap format, in
accordance with Juan's suggestion I have introduced a new command-line
option to explicitly specify the perfmap format revision. As of today,
0 corresponds to the legacy Crossgen1-style output where the
perfmap file name includes the {MVID} section, perfmap format #1
corresponds to current Crossgen2 with its new naming scheme.
As of today there are no differences in the file content.

Thanks

Tomas
adamsitnik pushed a commit that referenced this pull request Jan 13, 2022
…otnet#63598)

* Fix native frame unwind in syscall on arm64 for VS4Mac crash report.

Add arm64 version of StepWithCompactNoEncoding for syscall leaf node wrappers that have compact encoding of 0.

Fix ReadCompactEncodingRegister so it actually decrements the addr.

Change StepWithCompactEncodingArm64 to match what MacOS libunwind does for framed and frameless stepping.

arm64 can have frames with the same SP (but different IPs). Increment SP for this condition so createdump's unwind
loop doesn't break out on the "SP not increasing" check and the frames are added to the thread frame list in the
correct order.

Add getting the unwind info for tail called functions like this:

__ZL14PROCEndProcessPvji:
   36630:       f6 57 bd a9     stp     x22, x21, [sp, #-48]!
   36634:       f4 4f 01 a9     stp     x20, x19, [sp, #16]
   36638:       fd 7b 02 a9     stp     x29, x30, [sp, dotnet#32]
   3663c:       fd 83 00 91     add     x29, sp, dotnet#32
...
   367ac:       e9 01 80 52     mov     w9, #15
   367b0:       7f 3e 02 71     cmp     w19, dotnet#143
   367b4:       20 01 88 1a     csel    w0, w9, w8, eq
   367b8:       2e 00 00 94     bl      _PROCAbort
_TerminateProcess:
-> 367bc:       22 00 80 52     mov     w2, #1
   367c0:       9c ff ff 17     b       __ZL14PROCEndProcessPvji

The IP (367bc) returns the (incorrect) frameless encoding with nothing on the stack (uses an incorrect LR to unwind). To fix this
get the unwind info for PC -1 which points to PROCEndProcess with the correct unwind info. This matches how lldb unwinds this frame.

Always address module segment to IP lookup list instead of checking the module regions.

Strip pointer authentication bits on PC/LR.
adamsitnik pushed a commit that referenced this pull request Jan 25, 2022
* Fix the MacOS remote unwinder for VS4Mac

The wrong module was being passed to the remote unwinder because the load bias for shared modules
was being calculated incorrectly.

Issue: dotnet#63309

* Fix native frame unwind in syscall on arm64 for VS4Mac crash report

From PR in main: dotnet#63598

Add arm64 version of StepWithCompactNoEncoding for syscall leaf node wrappers that have compact encoding of 0.

Fix ReadCompactEncodingRegister so it actually decrements the addr.

Change StepWithCompactEncodingArm64 to match what MacOS libunwind does for framed and frameless stepping.

arm64 can have frames with the same SP (but different IPs). Increment SP for this condition so createdump's unwind
loop doesn't break out on the "SP not increasing" check and the frames are added to the thread frame list in the
correct order.

Add getting the unwind info for tail called functions like this:

__ZL14PROCEndProcessPvji:
   36630:       f6 57 bd a9     stp     x22, x21, [sp, #-48]!
   36634:       f4 4f 01 a9     stp     x20, x19, [sp, #16]
   36638:       fd 7b 02 a9     stp     x29, x30, [sp, dotnet#32]
   3663c:       fd 83 00 91     add     x29, sp, dotnet#32
...
   367ac:       e9 01 80 52     mov     w9, #15
   367b0:       7f 3e 02 71     cmp     w19, dotnet#143
   367b4:       20 01 88 1a     csel    w0, w9, w8, eq
   367b8:       2e 00 00 94     bl      _PROCAbort
_TerminateProcess:
-> 367bc:       22 00 80 52     mov     w2, #1
   367c0:       9c ff ff 17     b       __ZL14PROCEndProcessPvji

The IP (367bc) returns the (incorrect) frameless encoding with nothing on the stack (uses an incorrect LR to unwind). To fix this
get the unwind info for PC -1 which points to PROCEndProcess with the correct unwind info. This matches how lldb unwinds this frame.

Always address module segment to IP lookup list instead of checking the module regions.

Strip pointer authentication bits on PC/LR.
adamsitnik pushed a commit that referenced this pull request Jun 20, 2022
This adds support for EnC on arm64. A couple of notes on the
implementation compared to x64:
- On x64 we get the fixed stack size from unwind info. However, for the
  frames we set up on arm64 for EnC it is not possible to extract the
  frame size from there because their prologs generally look like

  stp fp, lr, [sp,#-16]!
  mov fp, sp
  sub sp, sp, dotnet#144

  with unwind codes like the following:

  set_fp; mov fp, sp

  save_fplr_x #1 (0x01); tp fp, lr, [sp, #-16]!

  As can be seen, it is not possible to get the fixed stack size from
  unwind info in this case. Instead we pass it through the GC info that
  already has a section for EnC data.

- On arm64 the JIT is required to place the PSPSym at the same offset
  from caller-SP for both the main function and for funclets. Due to
  this we try to allocate the PSPSym as early as possible in the main
  function and we must take some care in funclets.  However, this
  conflicts with the EnC frame header that the JIT uses to place values
  that must be preserved on EnC transitions. This is currently
  callee-saved registers and the MonitorAcquired boolean.

  Before this change we were allocating PSPSym above (before) the
  monitor acquired boolean, but we now have to allocate MonitorAcquired
  first, particularly because the size of the preserved header cannot
  change on EnC transitions, while the PSPSym can disappear or appear.
  This changes frame allocation slightly for synchronized functions.
adamsitnik pushed a commit that referenced this pull request Jun 20, 2022
These helpers are used to report names of things in warnings. The functional changes are:
* For method parameters, use the parameter name if available (and only if not fallback to the #1 notation)
* For property accessor methods, use the C# naming scheme, so for example Type.Property.get instead of Type.get_Property.

Both of these changes are in preparation to bring NativeAOT closer in behavior to ILLink and the trim analyzers.

For this I moved some of the helpers to the common shared code.

Some unrelated code cleanup as well.

Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>
adamsitnik pushed a commit that referenced this pull request Jun 23, 2022
* Initial implementation for contract customization

fix build errors

Move converter rooting to DefaultJsonTypeInfoResolver so that it can be used standalone

Fix ConfigurationList.IsReadOnly

Minor refactorings (#1)

* Makes the following changes:

* Move singleton initialization for DefaultTypeInfoResolver behind a static property.
* Consolidate JsonSerializerContext & IJsonTypeInfoResolver values to a single field.
* Move reflection fallback logic away from JsonSerializerContext and into JsonSerializerOptions

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializerOptions.cs

* remove testing of removed field

Simplify the JsonTypeInfo.CreateObject implemenetation (#2)

* Simplify the JsonTypeInfo.CreateObject implemenetation

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Tests and fixes for JsonTypeInfoKind.None

TypeInfo type mismatch tests

Allow setting NumberHandling on JsonTypeInfoKind.None

test resolver returning wrong type of options

JsonTypeInfo/JsonPropertyInfo mutability tests

rename test file

Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver (#3)

* Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver

* address feedback

Add simple test for using JsonTypeInfo<T> with APIs directly taking it

fix and tests for untyped/typed CreateObject

uncomment test cases, remove todo

More tests and tiny fixes

Add a JsonTypeInfoResolver.Combine test for JsonSerializerContext (#4)

* Fix JsonTypeInfoResolver.Combine for JsonSerializerContext

* Break up failing test

Fix simple scenarios for combining contexts (#6)

* Fix simple scenarios for combining contexts

* feedback

JsonSerializerContext combine test with different camel casing

Remove unneeded virtual calls & branching when accessing Get & Set delegates (#7)

JsonPropertyInfo tests everything minus ShouldSerialize & NumberHandling

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

throw InvalidOperationException rather than ArgumentNullException for source gen when PropertyInfo.Name is assigned through JsonPropertyInfoValues

tests for duplicated property names and JsonPropertyInfo.NumberHandling

Add tests for NumberHandling and failing tests for ShouldSerialize

disable the failing test and add extra checks

disable remainder of the failing ShouldSerialize tests, fix working one

Fix ShouldSerialize and IgnoreCondition interop

Add failing tests for CreateObject + parametrized constructors

Fix CreateObject support for JsonConstructor types (#10)

* Fix CreateObject support for JsonConstructor types

* address feedback

Make contexts more combinator friendly (#9)

* Make contexts more combinator friendly

* remove converter cache

* redesign test to account for JsonConstructorAttribute

* Combine unit tests

* address feedback

* Add acceptance tests for DataContract attributes & Specified pattern (#11)

* Add private field serialization acceptance test (#13)

* tests, PR feedback (#14)

* PR feedback and extra tests

* Shorten class name, remove incorrect check (not true for polimorphic cases)

* Make parameter matching for custom properties map property Name with parameter (#16)

* Test static initialization with JsonTypeInfo (dotnet#17)

* Fix test failures and proper fix this time (dotnet#18)

* Fix test failures and proper fix this time

* reinstate ActiveIssueAttribute

* PR feedback and adjust couple of tests which don't set TypeInfoResolver

* fix IAsyncEnumerable tests

* Lock JsonSerializerOptions in JsonTypeInfo.EnsureConfigured()

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
adamsitnik pushed a commit that referenced this pull request Jul 14, 2022
E.g.,

Update LSRA "Allocating Registers" table description.

Dump nodes added during resolution, e.g.:
```
   BB29 bottom (BB08->BB08): move V25 from STK to rdi (Critical)
N001 (  1,  1) [001174] ----------z                 t1174 =    LCL_VAR   int    V25 cse4          rdi REG rdi
```

Dump more data in the LSRA block sequence data:
```
-BB03( 16   )
-BB04(  4   )
+BB03 ( 16   ) critical-in critical-out
+BB04 (  4   ) critical-out
```

When dumping various flow bitvectors, annotate the bitvectors better:
```
-BB25 in gen out
-0000000000000000
-0000000000000003 CSE #1.c
-0000000000000003 CSE #1.c
+BB25
+ in: 0000000000000000
+gen: 0000000000000003 CSE #1.c
+out: 0000000000000003 CSE #1.c
```

Dump hoisting bitvectors using the sorting number:
```
-  USEDEF  (5)={V04 V00 V01 V02 V03}
+  USEDEF  (5)={V00 V01 V02 V03 V04}
```

Also, fix various typos and formatting.
adamsitnik pushed a commit that referenced this pull request Aug 25, 2022
* WIP: add gRPC tests

* Fix AOT and trimming

* WIP

* Implement IncludeNetworkSecurityConfig

* Use IncludeNetworkSecurityConfig

* Fix gRPC test

* Avoid git checkout

* Remove unnecessary code

* WIP: start working on CI configuration

* Remove WinHttpHandler

* Fix problem with SSL

* Change server host

* Setup CI (#1)

* Get Docker container building & exported via test build

* Changes

* Add missing pfx certificate

* changes

* cleanup

Co-authored-by: Simon Rozsival <simon@rozsival.com>

* Use tls

* Update yml

* Revert changes to the mono Android sample app

* Bump android image version

* Bump image version

* Enable TLS

* Remove hardcoded package versions

* Update package versions

* Update package versions

* Rename pipeline

* Move interop tests website dependencies versions to Versions.props

* Add cred scan supression for the interop test server private key

* Fix licenses

* Remove dependencies

* Fix path to Versions.props

* Remove unnecessary dependency version

* Fix building docker image

* Change pfx password

Co-authored-by: Jo Shields <directhex@apebox.org>
adamsitnik pushed a commit that referenced this pull request Dec 21, 2022
* switch to managed thread ID in Lock

* fattening the lock

* __declspec(selectany)

* few tweaks

* fairness

* more room for thread ids

* remove CurrentNativeThreadId

* couple fixes

* fix win-arm64 build

* win-arm64 build , another try

* Apply suggestions from code review

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

* fix after renaming

* do not report successful spin if thread has waited

* keep extern and undo mangling of tls_CurrentThread in asm

* use SyncTable indexer in less perf-sensitive places.

* GetNewHashCode just delegate to shared random

* Apply suggestions from code review

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

* unchecked const conversion

* some refactoring comments and typos

* min number of spins in the backoff

* moved CurrentManagedThreadIdUnchecked to ManagedThreadId

* Use `-1` to report success and allow using  element #1 in the SyncTable

* use threadstatic for managed thread ID

* check before calling RhGetProcessCpuCount

* use 0 as default thread ID

Co-authored-by: Jan Kotas <jkotas@microsoft.com>
adamsitnik pushed a commit that referenced this pull request Jun 12, 2023
…tnet#87189)

This fixes a startup crash on Big Sur:

> error: * Assertion at /Users/runner/work/1/s/src/mono/mono/utils/mono-hwcap-arm64.c:35, condition `res == 0' not met

Because sysctl can't find some of these options:

    $ sysctl hw.optional.armv8_crc32
    hw.optional.armv8_crc32: 1
    $ sysctl hw.optional.arm.FEAT_RDM
    sysctl: unknown oid 'hw.optional.arm.FEAT_RDM'
    $ sysctl hw.optional.arm.FEAT_DotProd
    sysctl: unknown oid 'hw.optional.arm.FEAT_DotProd'
    $ sysctl hw.optional.arm.FEAT_SHA1
    sysctl: unknown oid 'hw.optional.arm.FEAT_SHA1'
    $ sysctl hw.optional.arm.FEAT_SHA256
    sysctl: unknown oid 'hw.optional.arm.FEAT_SHA256'
    $ sysctl hw.optional.arm.FEAT_AES
    sysctl: unknown oid 'hw.optional.arm.FEAT_AES'

Full stack trace:

* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.1
  * frame #0: 0x0000010ef37560 libmonosgen-2.0.dylib`monoeg_assertion_message
    frame #1: 0x0000010ef375cc libmonosgen-2.0.dylib`mono_assertion_message + 32
    frame #2: 0x0000010ef40d6c libmonosgen-2.0.dylib`mono_hwcap_arch_init + 544
    frame #3: 0x0000010ef54bd8 libmonosgen-2.0.dylib`mono_hwcap_init + 72
    frame #4: 0x0000010ee14dc0 libmonosgen-2.0.dylib`parse_optimizations + 52
    frame #5: 0x0000010edbed48 libmonosgen-2.0.dylib`mono_init
    frame #6: 0x0000010ee18968 libmonosgen-2.0.dylib`mono_jit_init_version
    frame #7: 0x0000010f48a300 libxamarin-dotnet-debug.dylib`xamarin_bridge_initialize + 216
    frame #8: 0x0000010f4900a4 libxamarin-dotnet-debug.dylib`xamarin_main + 376
adamsitnik pushed a commit that referenced this pull request Jan 9, 2024
Fixes dotnet#95367.

Relevant part of the JitDump:

```
Using `if true` assertions from pred BB02
Assertions in: #1
fgMorphTree BB04, STMT00021 (before)
               [000070] DA---------                         *  STORE_LCL_VAR ubyte  V10 tmp9
               [000057] -----------                         \--*  CAST      int <- ubyte <- int
               [000006] -----------                            \--*  EQ        int
               [000004] -----------                               +--*  LCL_VAR   ref    V02 tmp1          (last use)
               [000055] H----------                               \--*  CNS_INT(h) ref     'Frozen EmptyPartition`1<Int32> object'

Assertion prop for index #1 in BB04:
               [000006] -----------                         *  EQ        int
GenTreeNode creates assertion:
               [000070] DA---+-----                         *  STORE_LCL_VAR ubyte  V10 tmp9
In BB04 New Local Constant Assertion: V10 == [0000000000000001], index = #2

fgMorphTree BB04, STMT00021 (after)
               [000070] DA---+-----                         *  STORE_LCL_VAR ubyte  V10 tmp9
               [000055] H----+-----                         \--*  CNS_INT(h) int
```

The JitDump is unfinished because the compiler crashes when trying to dump the last line. Clearly, the `CNS_INT` is no longer a handle at that point because we just bashed it to a constant 1.
@github-actions github-actions bot locked and limited conversation to collaborators Jan 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants