Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebGL crash with Meet VideoKit 0.0.10 - MediaDeviceQuery_DiscoverDevices #17

Closed
tomaszwozniak123 opened this issue Feb 24, 2023 · 10 comments
Labels
bug Something isn't working web Issues that affect Web

Comments

@tomaszwozniak123
Copy link

tomaszwozniak123 commented Feb 24, 2023

Hi,
In WebGL build sometimes (very often) when loading scene with NatML Meet Segmentation, there is a crash.
Please take a look in the Profiler screenshot - seems like MLModelData.OnEnable() drops FPS suddenly, and maybe this is a cause of crash? If not - could you suggest some solution for this? Is there a way to improve memory size for WebGL?
Or maybe it is directly connected with MediaDeviceQuery_DiscoverDevices?

This error occurs:
testtest.framework.js:3 Uncaught RuntimeError: memory access out of bounds at dlfree (testtest.wasm:0x9b73e) at MemoryManager::LowLevelFree(void*, unsigned long) (testtest.wasm:0x2c6f13) at UnityDefaultAllocator<LowLevelAllocator>::Deallocate(void*) (testtest.wasm:0x18a6591) at ThreadsafeLinearAllocator<false>::TryDeallocate(void*) (testtest.wasm:0x699528) at MemoryManager::Deallocate(void*, MemLabelId, char const*, int) (testtest.wasm:0x4a2737) at operator delete(void*) (testtest.wasm:0x9125e) at NDMediaDeviceGetName (testtest.wasm:0xf193bd) at NatDevice_Name_m7BE4A206300A690F90B92697C3E540277F177766 (testtest.wasm:0x7438fc) at AudioDevice__ctor_mE8A1C592C648F889936191E95E8E557A845B30E6 (testtest.wasm:0x117f137) at U3CDiscoverAudioDevicesU3Ed__20_MoveNext_m3947F1662EDDA93971217F8C7E67172E63FDE931 (testtest.wasm:0x1979979) at InterfaceFuncInvoker0<Toggle_tBF13F3EBA485E06826FD8A38F4B4C1380DF21A1F*>::Invoke(unsigned short, Il2CppClass*, Il2CppObject*) (testtest.wasm:0x918b5) at dynCall_iiii (testtest.wasm:0x1749226) at invoke_iiii (testtest.framework.js:3:525248) at List_1_AddEnumerable_mFD4E2919839A27C7CEACADF62222C63701302D02_gshared (testtest.wasm:0x1e81a36) at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_VoidU2A_t104EAEFBD2D237A8C29618913DA9B4D99355E965(void (*)(), MethodInfo const*, void*, void**, void*) (testtest.wasm:0xdf43cc) at InvokerActionInvoker1<RtcStats_tB82CEAB4F971E9D092B96B90C0C3BFFB035D1BE8>::Invoke(void (*)(), MethodInfo const*, void*, RtcStats_tB82CEAB4F971E9D092B96B90C0C3BFFB035D1BE8) (testtest.wasm:0x9292a) at List_1_InsertRange_m729639E2C8D1AA7579AC78D259085921E7DC72F2_gshared (testtest.wasm:0xf5b303) at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915_CancellationToken_t51142D9C6D7C02D314DA34A6A7988C528992FFED_RuntimeObject(void (*)(), MethodInfo const*, void*, void**, void*) (testtest.wasm:0x1c3d9d) at InvokerActionInvoker2<int, String_t*>::Invoke(void (*)(), MethodInfo const*, void*, int, String_t*) (testtest.wasm:0xa0b5a) at List_1_AddRange_m44DED39FFEBC8F2ACF6162DB065B649E888B9C57_gshared (testtest.wasm:0xf5ba9a) at List_1_AddRange_m66D40F60D60CF3FA653432ABEBEBF703840689DB(List_1_t437B6C3879E969156A381BDC3C459CF809D39DDF*, Il2CppObject*, MethodInfo const*) (testtest.wasm:0xa11cd) at MediaDeviceQuery_DiscoverDevices_m0520738B6315FF77EFB342F12AB3CF55102140C5 (testtest.wasm:0xf450aa) at MediaDeviceQuery__ctor_mE2C86815B53C8771C6F5BD82319E56505C59FF82 (testtest.wasm:0x2c74bc) at U252EU252DU252DU252EU252EU252EU252DU252DU252DU252EU252DU252DU252DU252EU252DU252DU252DU252EU252EU252EU252EU252DU252DU252EU252DU252EU252DU252DU252EU252EU252EU252EU252EU252DU252EU252EU252EU252DU252EU252EU252EU252EU252EU252DU252DU252EU252E_U252EU252EU252DU252DU252DU252EU252DU252EU252DU252EU252DU252EU252EU252EU252DU252DU252EU252EU252EU252EU252DU252EU252EU252EU252DU252DU252EU252EU252EU252EU252DU252EU252DU252DU252DU252DU252EU252DU252DU252EU252DU252DU252EU252DU252DU252EU252E_mA165F6D2E8CE5BDA1F792073168AB4F5C78B81EB (testtest.wasm:0x682594) at NatMlMeetBackgroundRemover_SwitchCameraDeviceToTarget_mF727B58824BDA93FF90F16C8943B75A8A71D35A5 (testtest.wasm:0x186543) at HallwayViewModel_U252DU252DU252EU252EU252EU252EU252EU252DU252EU252EU252DU252DU252EU252DU252EU252DU252EU252EU252EU252DU252DU252DU252DU252EU252DU252EU252DU252DU252EU252EU252EU252DU252DU252DU252DU252EU252DU252EU252EU252DU252EU252DU252DU252DU252EU252DU252E_mBED46317D9DAB642EACBDF579B8CC8D473313E97 (testtest.wasm:0x783e11) at HallwayViewModel_U252EU252EU252EU252EU252EU252DU252DU252EU252DU252DU252DU252EU252DU252DU252DU252DU252EU252DU252DU252DU252DU252EU252EU252EU252EU252DU252EU252EU252EU252DU252DU252EU252EU252DU252EU252EU252EU252DU252DU252DU252EU252EU252DU252EU252EU252DU252E_mE7C7F9E1E9446C4ECB510F50DDA0C22E8FE59917 (testtest.wasm:0xfe70f4) at HallwayViewModel_Start_mD1BD1C85DC59E11053F9FD98C15301F462C6A8B8 (testtest.wasm:0x1f49417) at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915(void (*)(), MethodInfo const*, void*, void**, void*) (testtest.wasm:0x1c82f76) at il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) (testtest.wasm:0x40d8b2) at dynCall_iiii (testtest.wasm:0x1749226) at invoke_iiii (testtest.framework.js:3:525248) at il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (testtest.wasm:0xf56fe) at il2cpp_runtime_invoke (testtest.wasm:0xaef264) at scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (testtest.wasm:0x3c948a) at ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (testtest.wasm:0x1c30a1) at ScriptingInvocation::Invoke(ScriptingExceptionPtr*) (testtest.wasm:0xf60f9) at MonoBehaviour::InvokeMethodOrCoroutineChecked(ScriptingMethodPtr, ScriptingObjectPtr) (testtest.wasm:0x7473aa) at MonoBehaviour::Start() (testtest.wasm:0x36b6c3) at MonoBehaviour::DelayedStartCall(Object*, void*) (testtest.wasm:0x1e2128e) at DelayedCallManager::Update(int) (testtest.wasm:0x2059dc9) at InitPlayerLoopCallbacks()::EarlyUpdateScriptRunDelayedStartupFrameRegistrator::Forward() (testtest.wasm:0x1f0da91) at ExecutePlayerLoop(NativePlayerLoopSystem*) (testtest.wasm:0x103f5a0) at ExecutePlayerLoop(NativePlayerLoopSystem*) (testtest.wasm:0x103f614) at MainLoop() (testtest.wasm:0xf50fda) at dynCall_v (testtest.wasm:0x1748a79) at browserIterationFunc (testtest.framework.js:3:257137) at callUserCallback (testtest.framework.js:3:210100) at Object.runIter (testtest.framework.js:3:211360) at Browser_mainLoop_runner (testtest.framework.js:3:209635)

image

@tomaszwozniak123
Copy link
Author

tomaszwozniak123 commented Feb 24, 2023

I have found a second error, which sometimes shows at the same moment as the one above:
testgoo.framework.js:3 Uncaught TypeError: Cannot read properties of null (reading 'remove') at methodCaller_void_$$ (eval at new_ (testgoo.framework.js:3:197163), <anonymous>:4:20) at __emval_call_void_method (testgoo.framework.js:3:196181) at NatML::Devices::CameraDevice::StopRunning() (testgoo.wasm:0x1e13499) at NDMediaDeviceStopRunning (testgoo.wasm:0x49cc2b) at CameraDevice_StopRunning_mD3E4E4E031EDFD4EDCFEF584F12C41C0C3131E82 (testgoo.wasm:0x8009e6) at VideoKitCameraManager_StopRunning_m3EC6267DD56C025F8E956F7659086DA23D04CBD3 (testgoo.wasm:0x1578f4) at VideoKitCameraManager_OnDestroy_m6818EA551700A0A76729AE0AD07E8812281A81F5 (testgoo.wasm:0x1846ca7) at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915(void (*)(), MethodInfo const*, void*, void**, void*) (testgoo.wasm:0x1c91be2) at il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) (testgoo.wasm:0x40d902) at dynCall_iiii (testgoo.wasm:0x1754f21) at invoke_iiii (testgoo.framework.js:3:525248) at il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (testgoo.wasm:0xf6224) at il2cpp_runtime_invoke (testgoo.wasm:0xaf0dde) at scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (testgoo.wasm:0x3c97f1) at ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (testgoo.wasm:0x1c79ca) at ScriptingInvocation::Invoke(ScriptingExceptionPtr*) (testgoo.wasm:0xf6ac5) at SerializableManagedRef::CallMethod(Object&, ScriptingMethodPtr) (testgoo.wasm:0xdcbc77) at MonoBehaviour::CallMethodInactive(ScriptingMethodPtr) (testgoo.wasm:0x18a236) at MonoBehaviour::WillDestroyComponent() (testgoo.wasm:0x1e2a8d2) at PreDestroyRecursive(GameObject&, unsigned long*) (testgoo.wasm:0xe63c24) at DestroyObjectHighLevel(Object*, bool) (testgoo.wasm:0x126c34) at UnloadGameScene() (testgoo.wasm:0xc0a75a) at LoadSceneOperation::PlayerLoadSceneFromThread() (testgoo.wasm:0xf6061d) at LoadSceneOperation::IntegrateMainThread() (testgoo.wasm:0x1e879eb) at PreloadManager::UpdatePreloadingSingleStep(PreloadManager::UpdatePreloadingFlags, int) (testgoo.wasm:0xe62d5b) at PreloadManager::UpdatePreloading() (testgoo.wasm:0xe62b7c) at InitPlayerLoopCallbacks()::EarlyUpdateUpdatePreloadingRegistrator::Forward() (testgoo.wasm:0x1f168bc) at ExecutePlayerLoop(NativePlayerLoopSystem*) (testgoo.wasm:0x1048343) at ExecutePlayerLoop(NativePlayerLoopSystem*) (testgoo.wasm:0x10483b7) at MainLoop() (testgoo.wasm:0xf586a6) at dynCall_v (testgoo.wasm:0x1754774) at browserIterationFunc (testgoo.framework.js:3:257137) at callUserCallback (testgoo.framework.js:3:210100) at Object.runIter (testgoo.framework.js:3:211360) at Browser_mainLoop_runner (testgoo.framework.js:3:209635)

It points to this part of code in VideoKitCameraManager:

public void StopRunning () { cameraDevice?.StopRunning(); output?.Dispose(); output = null; }

And in addition I see this warning:

image

@tomaszwozniak123 tomaszwozniak123 changed the title WebGL crash with Meet VideoKit 0.0.10 WebGL crash with Meet VideoKit 0.0.10 - MediaDeviceQuery_DiscoverDevices Feb 24, 2023
@olokobayusuf olokobayusuf added web Issues that affect Web bug Something isn't working labels Feb 25, 2023
@olokobayusuf
Copy link
Contributor

In WebGL build sometimes (very often) when loading scene with NatML Meet Segmentation, there is a crash.

The call stack seems to indicate something about getting the name of the camera device. What browser and Unity versions are you on?

have found a second error, which sometimes shows at the same moment as the one above:

This is related to the first, and happens because the camera preview is being stopped, but due to the first error, the camera is an invalid state.

@tomaszwozniak123
Copy link
Author

Hi
Browser version: Chrome Version 110.0.5481.105 (Official Build) (64-bit)
Unity: 2021.3.12f1

@tomaszwozniak123
Copy link
Author

tomaszwozniak123 commented Feb 27, 2023

Additionally, here are my methods used with MediaDeviceQuery, maybe it helps in investigation:

        public static class VideoKitCameraManagerCustom
        {
            public static CameraDevice GetNextCameraDevice(this VideoKitCameraManager videoKitCameraManager, CameraDevice currentCamera)
            {
                MediaDeviceQuery.ConfigureAudioSession = false;
                var query = new MediaDeviceQuery(MediaDeviceCriteria.CameraDevice);
                var devices = query.OrderBy(device => device is CameraDevice camera).ToList();
                var currentDeviceIndex = devices.IndexOf(currentCamera);

            if (currentDeviceIndex < devices.Count - 1)
            {
                currentDeviceIndex++;
            }
            else
            {
                currentDeviceIndex = 0;
            }
            var nextDevice = devices[currentDeviceIndex];
            return (CameraDevice)nextDevice;
        }

        public static CameraDevice GetNextTargetCameraDevice(this VideoKitCameraManager videoKitCameraManager, int index)
        {
            MediaDeviceQuery.ConfigureAudioSession = false;
            var query = new MediaDeviceQuery(MediaDeviceCriteria.CameraDevice);
            var devices = query.OrderBy(device => device is CameraDevice camera).ToList();
            var nextDevice = devices[Mathf.Max(0, index)];
            if (nextDevice != null)
            {
                return (CameraDevice)nextDevice;
            }
            else
            {
                return (CameraDevice)devices[0];
            }
        }

        public static int GetCurrentCameraDeviceIndex(this VideoKitCameraManager videoKitCameraManager, CameraDevice currentCamera)
        {
            MediaDeviceQuery.ConfigureAudioSession = false;
            var query = new MediaDeviceQuery(MediaDeviceCriteria.CameraDevice);
            var devices = query.OrderBy(device => device is CameraDevice camera).ToList();
            var currentDeviceIndex = devices.IndexOf(currentCamera);
            return currentDeviceIndex;
        }
    }

@olokobayusuf
Copy link
Contributor

Hey @tomaszwozniak123 , can you try VideoKit 0.0.11?

@tomaszwozniak123
Copy link
Author

tomaszwozniak123 commented Mar 2, 2023

Hey @olokobayusuf ,
I used VideoKit 0.0.11 and get this error instead previous ones (it happens only in browser-Unity Editor works fine). It happens when trying switching scenes:

testgoo.loader.js:1 exception thrown: TypeError: Cannot read properties of null (reading 'remove'),TypeError: Cannot read properties of null (reading 'remove')
    at methodCaller_void_$$ (eval at new_ (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.framework.js:3:197163), <anonymous>:4:20)
    at __emval_call_void_method (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.framework.js:3:196181)
    at NatML::Devices::CameraDevice::StopRunning() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[116334]:0x25f2b51)
    at CameraDevice_StopRunning_mD3E4E4E031EDFD4EDCFEF584F12C41C0C3131E82 (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[30092]:0xa53001)
    at VideoKitCameraManager_StopRunning_m3EC6267DD56C025F8E956F7659086DA23D04CBD3 (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[5809]:0x1bbebe)
    at VideoKitCameraManager_OnDestroy_m6818EA551700A0A76729AE0AD07E8812281A81F5 (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[79002]:0x1d97e4f)
    at RuntimeInvoker_TrueVoid_t4861ACF8F4594C3437BB48B6E56783494B843915(void (*)(), MethodInfo const*, void*, void**, void*) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[106031]:0x23a4543)
    at il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const*, void*, void**) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[16615]:0x52ee7f)
    at dynCall_iiii (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[73323]:0x1c3cf3a)
    at invoke_iiii (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.framework.js:3:526839)
    at il2cpp::vm::Runtime::Invoke(MethodInfo const*, void*, void**, Il2CppException**) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[3695]:0x13f753)
    at il2cpp_runtime_invoke (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[34461]:0xd99a8f)
    at scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr*, bool) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[16173]:0x4e91ca)
    at ScriptingInvocation::Invoke(ScriptingExceptionPtr*, bool) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[1685]:0xe2e5d)
    at ScriptingInvocation::InvokeChecked(ScriptingExceptionPtr*) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[44139]:0x113695f)
    at SerializableManagedRef::CallMethod(Object&, ScriptingMethodPtr) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[6732]:0x1f760f)
    at MonoBehaviour::WillDestroyComponent() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[116853]:0x260e64e)
    at PreDestroyRecursive(GameObject&, unsigned long*) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[46575]:0x11f2abf)
    at DestroyObjectHighLevel_Internal(Object*, bool) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[29184]:0x9f9a4e)
    at UnloadGameScene() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[38571]:0xefa508)
    at LoadSceneOperation::PlayerLoadSceneFromThread() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[48992]:0x12ed739)
    at LoadSceneOperation::IntegrateMainThread() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[118849]:0x26899d6)
    at PreloadManager::UpdatePreloadingSingleStep(PreloadManager::UpdatePreloadingFlags, int) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[46527]:0x11f0b7d)
    at PreloadManager::UpdatePreloading() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[46526]:0x11f090c)
    at InitPlayerLoopCallbacks()::EarlyUpdateUpdatePreloadingRegistrator::Forward() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[123766]:0x27bce43)
    at ExecutePlayerLoop(NativePlayerLoopSystem*) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[54795]:0x143cde2)
    at ExecutePlayerLoop(NativePlayerLoopSystem*) (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[54795]:0x143ce53)
    at MainLoop() (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[48815]:0x12e476a)
    at dynCall_v (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.wasm:wasm-function[73262]:0x1c3c735)
    at browserIterationFunc (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.framework.js:3:257137)
    at callUserCallback (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.framework.js:3:210100)
    at Object.runIter (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.framework.js:3:211360)
    at Browser_mainLoop_runner (https://v6p9d9t4.ssl.hwcdn.net/html/7446927/Build/testgoo.framework.js:3:209635)

@tomaszwozniak123
Copy link
Author

tomaszwozniak123 commented Mar 2, 2023

Hey @olokobayusuf
Update for the error above:
When I added these two condition checks to StopRunning() method in the CameraDevice class, it stopped crashing (haven't tested much, so please take it as a hint only):

 public void StopRunning () {
            if (running==false) { return; }
            if (device!=null)
            {
                device.StopRunning().CheckStatus();
            }

            lifecycleHelper?.Dispose();
            lifecycleHelper = default;
            if (previewHandle != default)
                previewHandle.Free();
            previewHandle = default;
        }

@olokobayusuf
Copy link
Contributor

Hey @tomaszwozniak123 sorry for the late follow up on this. Your fix is correct, and the fix is landing in 0.0.12.

@olokobayusuf
Copy link
Contributor

Fixed in VideoKit 0.0.12.

@tomaszwozniak123
Copy link
Author

Works great, thanks! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working web Issues that affect Web
Projects
None yet
Development

No branches or pull requests

2 participants