v0.4.0-rc2 Android Demo App for Llama crashing on Pixel 8 when first prompt submitted #5582

grisaitis · 2024-09-24T15:58:05Z

🐛 Describe the bug

I find that when I submit my first prompt on the demo app, the app crashes.

See this screenshot video:

first 30 seconds: model is loading (successfully i think! memory consumption goes up)
at 0:30, i type hello and press the enter button
then crashes

screen-20240924-174804.mp4

This is the logcat output:

---------------------------- PROCESS STARTED (29659) for package com.example.executorchllamademo ----------------------------
2024-09-24 17:46:45.765 29659-29659 ziparchive              com.example.executorchllamademo      W  Unable to open '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.dm': No such file or directory
2024-09-24 17:46:45.765 29659-29659 ziparchive              com.example.executorchllamademo      W  Unable to open '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.dm': No such file or directory
2024-09-24 17:46:45.927 29659-29659 nativeloader            com.example.executorchllamademo      D  Configuring clns-6 for other apk /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk. target_sdk_version=33, uses_libraries=, library_path=/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64:/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!/lib/arm64-v8a, permitted_path=/data:/mnt/expand:/data/user/0/com.example.executorchllamademo
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V  Currently set values for:
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V    angle_gl_driver_selection_pkgs=[com.android.angle, com.linecorp.b612.android, com.campmobile.snow, com.google.android.apps.tachyon]
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V    angle_gl_driver_selection_values=[angle, native, native, native]
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V  com.example.executorchllamademo is not listed in per-application setting
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V  Neither updatable production driver nor prerelease driver is supported.
2024-09-24 17:46:45.959 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 279646685; UID 10448; state: DISABLED
2024-09-24 17:46:46.018 29659-29709 vulkan                  com.example.executorchllamademo      D  searching for layers in '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64'
2024-09-24 17:46:46.018 29659-29709 vulkan                  com.example.executorchllamademo      D  searching for layers in '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!/lib/arm64-v8a'
2024-09-24 17:46:46.037 29659-29659 AppCompatDelegate       com.example.executorchllamademo      D  Checking for metadata for AppLocalesMetadataHolderService : Service not found
2024-09-24 17:46:46.075 29659-29659 utorchllamademo         com.example.executorchllamademo      W  Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (unsupported, reflection, allowed)
2024-09-24 17:46:46.075 29659-29659 utorchllamademo         com.example.executorchllamademo      W  Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (unsupported, reflection, allowed)
2024-09-24 17:46:46.079 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 210923482; UID 10448; state: ENABLED
2024-09-24 17:46:46.079 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 63938206; UID 10448; state: DISABLED
2024-09-24 17:46:46.093 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 171228096; UID 10448; state: ENABLED
2024-09-24 17:46:46.124 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 303326708; UID 10448; state: DISABLED
2024-09-24 17:46:46.158 29659-29659 ETLogging               com.example.executorchllamademo      D  To get started, select your desired model and tokenizer from the top right corner
2024-09-24 17:46:46.161 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 237531167; UID 10448; state: DISABLED
2024-09-24 17:46:46.337 29659-29664 utorchllamademo         com.example.executorchllamademo      I  Compiler allocated 4206KB to compile void android.widget.TextView.<init>(android.content.Context, android.util.AttributeSet, int, int)
2024-09-24 17:46:46.640 29659-29664 utorchllamademo         com.example.executorchllamademo      I  Compiler allocated 4666KB to compile void android.view.ViewRootImpl.performTraversals()
2024-09-24 17:46:51.929 29659-29799 ProfileInstaller        com.example.executorchllamademo      D  Installing profile for com.example.executorchllamademo
2024-09-24 17:47:06.899 29659-29665 utorchllamademo         com.example.executorchllamademo      I  Background concurrent mark compact GC freed 3046KB AllocSpace bytes, 2(40KB) LOS objects, 86% free, 3841KB/27MB, paused 323us,5.200ms total 59.694ms
2024-09-24 17:47:25.358 29659-29659 ETLogging               com.example.executorchllamademo      D  mModelType from settings LLAMA_3
2024-09-24 17:47:25.903 29659-29659 VRI[MainActivity]       com.example.executorchllamademo      D  visibilityChanged oldVisibility=true newVisibility=false
2024-09-24 17:47:25.916 29659-29709 HWUI                    com.example.executorchllamademo      D  endAllActiveAnimators on 0xb40000762225bdd0 (RippleDrawable) with handle 0xb4000074b22503e0
2024-09-24 17:47:28.139 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.159 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.176 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.193 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.210 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.226 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.931 29659-29659 WindowOnBackDispatcher  com.example.executorchllamademo      W  sendCancelIfRunning: isInProgress=falsecallback=android.view.ViewRootImpl$$ExternalSyntheticLambda11@efba2ce
2024-09-24 17:47:28.933 29659-29709 HWUI                    com.example.executorchllamademo      D  endAllActiveAnimators on 0xb40000762228c8c0 (RippleDrawable) with handle 0xb4000074b2241980
2024-09-24 17:47:28.980 29659-30259 ETLogging               com.example.executorchllamademo      D  Loading model /data/local/tmp/llama/llama_2_7B_chat.pte with tokenizer /data/local/tmp/llama/tokenizer_llama_2_7b_chat.bin
2024-09-24 17:47:28.999 29659-30259 libc                    com.example.executorchllamademo      W  Access denied finding property "ro.mediatek.platform"
2024-09-24 17:47:28.999 29659-30259 libc                    com.example.executorchllamademo      W  Access denied finding property "ro.chipname"
2024-09-24 17:47:28.999 29659-30259 libc                    com.example.executorchllamademo      W  Access denied finding property "ro.hardware.chipname"
2024-09-24 17:47:29.003 29659-30259 nativeloader            com.example.executorchllamademo      D  Load /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so using ns clns-6 from class loader (caller=/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!classes4.dex): ok
2024-09-24 17:47:29.026 29659-30259 nativeloader            com.example.executorchllamademo      D  Load /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libfbjni.so using ns clns-6 from class loader (caller=/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!classes4.dex): ok
2024-09-24 17:47:29.532 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  visibilityChanged oldVisibility=true newVisibility=false
2024-09-24 17:47:29.549 29659-29659 WindowOnBackDispatcher  com.example.executorchllamademo      W  sendCancelIfRunning: isInProgress=falsecallback=android.view.ViewRootImpl$$ExternalSyntheticLambda11@7f26428
2024-09-24 17:47:52.387 29659-30259 ETLogging               com.example.executorchllamademo      D  Load complete. Model path: /data/local/tmp/llama/llama_2_7B_chat.pte
                                                                                                    Tokenizer path: /data/local/tmp/llama/tokenizer_llama_2_7b_chat.bin
                                                                                                    Temperature: 0.1
                                                                                                    Model loaded time: 23405 ms
2024-09-24 17:47:52.421 29659-29659 utorchllamademo         com.example.executorchllamademo      I  Waiting for a blocking GC NativeAlloc
2024-09-24 17:47:52.495 29659-29665 utorchllamademo         com.example.executorchllamademo      W  Cleared Reference was only reachable from finalizer (only reported once)
2024-09-24 17:47:52.524 29659-29665 utorchllamademo         com.example.executorchllamademo      I  NativeAlloc concurrent mark compact GC freed 1308KB AllocSpace bytes, 7(268KB) LOS objects, 85% free, 4235KB/28MB, paused 354us,3.715ms total 103.740ms
2024-09-24 17:47:52.524 29659-29659 utorchllamademo         com.example.executorchllamademo      I  WaitForGcToComplete blocked NativeAlloc on NativeAlloc for 102.967ms
2024-09-24 17:47:55.174 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 163400105; UID 10448; state: ENABLED
2024-09-24 17:47:55.176 29659-29659 ImeTracker              com.example.executorchllamademo      I  com.example.executorchllamademo:980d09ca: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser true
2024-09-24 17:47:55.180 29659-29659 InputMethodManager      com.example.executorchllamademo      D  showSoftInput() view=androidx.appcompat.widget.AppCompatEditText{3efbcf2 VFED..CL. .F.P..ID 115,0-965,92 #7f0800b8 app:id/editTextMessage aid=1073741824} flags=0 reason=SHOW_SOFT_INPUT
2024-09-24 17:47:55.186 29659-29659 Choreographer           com.example.executorchllamademo      I  Skipped 80 frames!  The application may be doing too much work on its main thread.
2024-09-24 17:47:55.201 29659-29659 AssistStructure         com.example.executorchllamademo      I  Flattened final assist data: 3088 bytes, containing 1 windows, 17 views
2024-09-24 17:47:55.208 29659-29670 HWUI                    com.example.executorchllamademo      I  Davey! duration=1344ms; Flags=0, FrameTimelineVsyncId=151756, IntendedVsync=912293811709, Vsync=913625807069, InputEventId=-245770153, HandleInputStart=913628434812, AnimationStart=913628986529, PerformTraversalsStart=913628988156, DrawStart=913629254758, FrameDeadline=912310411709, FrameInterval=913627812375, FrameStartTime=16649942, SyncQueued=913632489784, SyncStart=913632581703, IssueDrawCommandsStart=913632703895, SwapBuffers=913636981198, FrameCompleted=913638822995, DequeueBufferDuration=25187, QueueBufferDuration=252319, GpuCompleted=913638822995, SwapBuffersCompleted=913637359209, DisplayPresentTime=881024008323, CommandSubmissionCompleted=913636981198, 
2024-09-24 17:47:55.398 29659-29659 InsetsController        com.example.executorchllamademo      D  show(ime(), fromIme=true)
2024-09-24 17:47:55.405 29659-30637 InteractionJankMonitor  com.example.executorchllamademo      W  Initializing without READ_DEVICE_CONFIG permission. enabled=false, interval=1, missedFrameThreshold=3, frameTimeThreshold=64, package=com.example.executorchllamademo
2024-09-24 17:47:55.637 29659-29659 ImeTracker              com.example.executorchllamademo      I  com.example.executorchllamademo:980d09ca: onShown
2024-09-24 17:48:00.109 29659-30900 ETLogging               com.example.executorchllamademo      D  starting runnable generate()
2024-09-24 17:48:00.109 29659-30900 ETLogging               com.example.executorchllamademo      D  Running inference.. prompt=<|begin_of_text|><|start_header_id|>system<|end_header_id|>
                                                                                                    thing<|eot_id|><|start_header_id|>user<|end_header_id|>
                                                                                                    hello<|eot_id|><|start_header_id|>assistant<|end_header_id|>
                                                                                                    <|eot_id|><|start_header_id|>user<|end_header_id|>
                                                                                                    hello<|eot_id|>
                                                                                                    <|start_header_id|>assistant<|end_uheader_id|>
2024-09-24 17:48:00.183 29659-30900 libc                    com.example.executorchllamademo      A  Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 30900 (pool-3-thread-1), pid 29659 (utorchllamademo)
---------------------------- PROCESS STARTED (30915) for package com.example.executorchllamademo ----------------------------
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A  Cmdline: com.example.executorchllamademo
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A  pid: 29659, tid: 30900, name: pool-3-thread-1  >>> com.example.executorchllamademo <<<
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #01 pc 0000000001b23e94  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (et_pal_abort+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #02 pc 0000000001b23ddc  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (executorch::runtime::runtime_abort()+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #03 pc 0000000001aa15d0  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (example::Runner::generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, int, std::__ndk1::function<void (std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&)>, std::__ndk1::function<void (executorch::extension::llm::Stats const&)>, bool)+2888) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #04 pc 0000000000189798  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (executorch_jni::ExecuTorchLlamaJni::generate(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char)+376) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #05 pc 0000000000189f1c  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (facebook::jni::detail::MethodWrapper<int (executorch_jni::ExecuTorchLlamaJni::*)(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char), &(executorch_jni::ExecuTorchLlamaJni::generate(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char)), executorch_jni::ExecuTorchLlamaJni, int, facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char>::dispatch(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, facebook::jni::alias_ref<_jstring*>&&, int&&, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>&&, unsigned char&&)+156) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #06 pc 0000000000189dfc  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (facebook::jni::detail::FunctionWrapper<int (*)(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, facebook::jni::alias_ref<_jstring*>&&, int&&, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>&&, unsigned char&&), facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*, int, facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char>::call(_JNIEnv*, _jobject*, _jintArray*, int, int, int, _jstring*, int, facebook::jni::detail::JTypeFor<executorch_jni::ExecuTorchLlamaCallbackJni, facebook::jni::JObject, void>::_javaobject*, unsigned char, int (*)(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, facebook::jni::alias_ref<_jstring*>&&, int&&, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>&&, unsigned char&&))+168) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #07 pc 00000000001886a4  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (facebook::jni::detail::MethodWrapper<int (executorch_jni::ExecuTorchLlamaJni::*)(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char), &(executorch_jni::ExecuTorchLlamaJni::generate(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char)), executorch_jni::ExecuTorchLlamaJni, int, facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char>::call(_JNIEnv*, _jobject*, _jintArray*, int, int, int, _jstring*, int, facebook::jni::detail::JTypeFor<executorch_jni::ExecuTorchLlamaCallbackJni, facebook::jni::JObject, void>::_javaobject*, unsigned char)+40) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #14 pc 00000000003571b4  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk (org.pytorch.executorch.LlamaModule.generate+0)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #19 pc 0000000000006ac0  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk (com.example.executorchllamademo.MainActivity$4.run+0)
---------------------------- PROCESS ENDED (30915) for package com.example.executorchllamademo ----------------------------
---------------------------- PROCESS ENDED (29659) for package com.example.executorchllamademo ----------------------------

Any idea why this is happening?

Versions

PyTorch version: 2.5.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.4.1 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: version 3.30.3
Libc version: N/A

Python version: 3.10.0 (default, Mar 3 2022, 03:54:28) [Clang 12.0.0 ] (64-bit runtime)
Python platform: macOS-14.4.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1

Versions of relevant libraries:
[pip3] executorch==0.4.0a0+759e0c8
[pip3] numpy==1.26.4
[pip3] torch==2.5.0
[pip3] torchaudio==2.5.0
[pip3] torchsr==1.0.4
[pip3] torchvision==0.20.0
[conda] executorch 0.4.0a0+759e0c8 pypi_0 pypi
[conda] numpy 1.26.4 pypi_0 pypi
[conda] torch 2.5.0 pypi_0 pypi
[conda] torchaudio 2.5.0 pypi_0 pypi
[conda] torchsr 1.0.4 pypi_0 pypi
[conda] torchvision 0.20.0 pypi_0 pypi

build settings for the app

i'm following these tutorial pages (note, these are from v0.4.0-rc2 release)

export ANDROID_HOME="/Users/william/Library/Android/sdk"
export ANDROID_NDK="/Users/william/Library/Android/sdk/ndk/25.2.9519653"
export ANDROID_ABI="arm64-v8a"

pushd extension/android                   
   ./gradlew build            
popd
pushd examples/demo-apps/android/LlamaDemo
   ./gradlew :app:setup       
popd
pushd examples/demo-apps/android/LlamaDemo                                            
   ./gradlew :app:installDebug
popd

The text was updated successfully, but these errors were encountered:

dvorjackz · 2024-09-24T18:46:59Z

cc @kirklandsign

kirklandsign · 2024-09-24T18:51:45Z

Hi @grisaitis may I ask which phone are you using and how many RAM is it?

It's not related to app/JNI, but runtime itself.

grisaitis · 2024-09-24T19:51:47Z

Sure - Google Pixel 8, with 8GB. Confirmed from my system settings.

Thanks so much for helping troubleshoot this! This library is amazing.

kirklandsign · 2024-09-24T20:13:39Z

@grisaitis Thank you for the info! Do you have any lmkd from logcat?

grisaitis · 2024-09-25T08:28:27Z

looks like it is a low memory issue, but i'm not sure. these are the lmkd logcat messages right after i click the enter arrow for a prompt:

2024-09-25 10:24:19.378   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.scone' (15964), uid 10137, oom_score_adj 500 to free 92212kB rss, 33036kB swap; reason: low watermark is breached and swap is low (712kB < 389076kB)
2024-09-25 10:24:21.869   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.carrier' (16153), uid 10113, oom_score_adj 905 to free 61004kB rss, 35280kB swap; reason: low watermark is breached and swap is low (0kB < 389076kB)
2024-09-25 10:24:21.887   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.garmin.android.apps.connectmobile' (15970), uid 10284, oom_score_adj 250 to free 178952kB rss, 32148kB swap; reason: low watermark is breached and swap is low (0kB < 389076kB)
2024-09-25 10:24:22.307   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.GoogleCamera' (16267), uid 10121, oom_score_adj 500 to free 107024kB rss, 34436kB swap; reason: low watermark is breached and swap is low (8kB < 389076kB)
2024-09-25 10:24:22.586   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.scone' (16277), uid 10137, oom_score_adj 250 to free 90028kB rss, 33128kB swap; reason: low watermark is breached and swap is low (36kB < 389076kB)
2024-09-25 10:24:22.642   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.chromecast.app' (16270), uid 10364, oom_score_adj 250 to free 140084kB rss, 34104kB swap; reason: low watermark is breached and swap is low (240kB < 389076kB)
2024-09-25 10:24:22.675   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.process.gservices' (16345), uid 10170, oom_score_adj 250 to free 68600kB rss, 34872kB swap; reason: low watermark is breached and swap is low (776kB < 389076kB)
2024-09-25 10:24:22.725   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.projection.gearhead:shared' (16309), uid 10159, oom_score_adj 905 to free 93804kB rss, 34260kB swap; reason: low watermark is breached and swap is low (52kB < 389076kB)
2024-09-25 10:24:22.814   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.carrier' (16472), uid 10113, oom_score_adj 905 to free 62372kB rss, 35496kB swap; reason: low watermark is breached and swap is low (0kB < 389076kB)
2024-09-25 10:24:22.815   495-500   lowmemorykiller         lmkd                                 E  process_mrelease 16472 failed: No such process
---------------------------- PROCESS STARTED (16387) for package com.example.executorchllamademo ----------------------------
2024-09-25 10:24:22.975   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.projection.gearhead:car' (16505), uid 10159, oom_score_adj 905 to free 89220kB rss, 34844kB swap; reason: low watermark is breached and swap is low (7436kB < 389076kB)
2024-09-25 10:24:22.998   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.wearables.maestro.companion' (16468), uid 10131, oom_score_adj 915 to free 81072kB rss, 34608kB swap; reason: low watermark is breached and swap is low (14652kB < 389076kB)
---------------------------- PROCESS ENDED (16387) for package com.example.executorchllamademo ----------------------------
---------------------------- PROCESS ENDED (15155) for package com.example.executorchllamademo ----------------------------

any additional tips appreciated!

grisaitis · 2024-09-25T08:39:29Z

PS here are all logcat messages, from adb logcat > logcat_output.txt

logcat_output.txt

i asked claude to analyze the output. it suggested the error was originating in the Runner::generate call in the native library, highlighting these three lines from the logcat output.

09-25 10:32:39.208 20618 20618 F DEBUG   :       #01 pc 0000000001b23e94  /data/app/~~-Z1bTFwXFcIAkEa8u0899w==/com.example.executorchllamademo-HVqGfQ-pjnutaF5__CmORw==/lib/arm64/libexecutorch.so (et_pal_abort+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
09-25 10:32:39.208 20618 20618 F DEBUG   :       #02 pc 0000000001b23ddc  /data/app/~~-Z1bTFwXFcIAkEa8u0899w==/com.example.executorchllamademo-HVqGfQ-pjnutaF5__CmORw==/lib/arm64/libexecutorch.so (executorch::runtime::runtime_abort()+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
09-25 10:32:39.208 20618 20618 F DEBUG   :       #03 pc 0000000001aa15d0  /data/app/~~-Z1bTFwXFcIAkEa8u0899w==/com.example.executorchllamademo-HVqGfQ-pjnutaF5__CmORw==/lib/arm64/libexecutorch.so (example::Runner::generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, int, std::__ndk1::function<void (std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&)>, std::__ndk1::function<void (executorch::extension::llm::Stats const&)>, bool)+2888) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)

any additional tips appreciated. perhaps i should build the executorch library with logging enabled? eg with -DEXECUTORCH_ENABLE_LOGGING=1 flags added to the cmake calls.

kirklandsign · 2024-09-25T17:29:24Z

Thank you! Seems like a low memory issue. For logging, please -DEXECUTORCH_ENABLE_LOGGING=1 and remove Release config https://github.com/pytorch/executorch/blob/main/build/build_android_llm_demo.sh#L47-L55

kirklandsign · 2024-09-25T17:42:57Z

@grisaitis The good news is you can try with the new LLAMA 3.2 1B model https://github.com/pytorch/executorch/pull/5640/files#diff-9a76949fefe609e9fcad45db3f4c6cb18d7913054d0122f672b30f523cadbc78R70

grisaitis · 2024-09-26T06:40:19Z

Thank you! Seems like a low memory issue. For logging, please -DEXECUTORCH_ENABLE_LOGGING=1 and remove Release config https://github.com/pytorch/executorch/blob/main/build/build_android_llm_demo.sh#L47-L55

thanks! i'll try this.

clarification - should i simply do the edit like this? removing the line for -DCMAKE_BUILD_TYPE=Release, and removing --config Release.

diff --git a/build/build_android_llm_demo.sh b/build/build_android_llm_demo.sh
index 42034c2..2feb48e 100644
--- a/build/build_android_llm_demo.sh
+++ b/build/build_android_llm_demo.sh
@@ -44,7 +44,6 @@ build_android_native_library() {
     -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
     -DEXECUTORCH_BUILD_QNN="${EXECUTORCH_BUILD_QNN}" \
     -DQNN_SDK_ROOT="${QNN_SDK_ROOT}" \
-    -DCMAKE_BUILD_TYPE=Release \
     -B"${CMAKE_OUT}"
 
   if [ "$(uname)" == "Darwin" ]; then
@@ -52,7 +51,7 @@ build_android_native_library() {
   else
     CMAKE_JOBS=$(( $(nproc) - 1 ))
   fi
-  cmake --build "${CMAKE_OUT}" -j "${CMAKE_JOBS}" --target install --config Release
+  cmake --build "${CMAKE_OUT}" -j "${CMAKE_JOBS}" --target install
 
   cmake extension/android \
     -DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK}/build/cmake/android.toolchain.cmake \

i will probably have to make these changes not in build/, but in the examples/**/*.sh files. when i attempt running build/build_android_llm_demo.sh, i get errors for missing env variables. i have more experience with the other build scripts. let me know if you think that approach (modifying examples/**/*.sh files and not build/build_android_llm_demo.sh) is still viable.

as for other models, i downloaded llama 3 8B and quantized it. i also started downloading 3.1 8B and 3.2 3B. will try those out later, if llama 3 8B doesn't work.

kirklandsign · 2024-09-26T06:44:56Z

i will probably have to make these changes not in build/, but in the examples//*.sh files. when i attempt running build/build_android_llm_demo.sh, i get errors for missing env variables. i have more experience with the other build scripts. let me know if you think that approach (modifying examples//*.sh files and not build/build_android_llm_demo.sh) is still viable.

That's correct. Remove -DCMAKE_BUILD_TYPE=Release \ and --config Release from the sh.
(though for me I still see logcat with Release build)

What missing env variables is it? If it's related to copying artifacts for upload please disregard :) I will clean up it later.

as for other models, i downloaded llama 3 8B and quantized it. i also started downloading 3.1 8B and 3.2 3B. will try those out later, if llama 3 8B doesn't work.

Maybe try 3.2 1B? It has lower RAM requirement

grisaitis · 2024-09-27T14:02:13Z

That's correct. Remove -DCMAKE_BUILD_TYPE=Release \ and --config Release from the sh.

thanks! i did this along with + -DEXECUTORCH_ENABLE_LOGGING=ON \ in examples/demo-apps/android/LlamaDemo/setup.sh, resulting in the following diff:

diff

diff --git a/examples/demo-apps/android/LlamaDemo/setup.sh b/examples/demo-apps/android/LlamaDemo/setup.sh
index b89c182..fa02c05 100644
--- a/examples/demo-apps/android/LlamaDemo/setup.sh
+++ b/examples/demo-apps/android/LlamaDemo/setup.sh
@@ -13,6 +13,7 @@ cmake . -DCMAKE_INSTALL_PREFIX="${CMAKE_OUT}" \
   -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \
   -DANDROID_ABI="${ANDROID_ABI}" \
   -DANDROID_PLATFORM=android-23 \
+  -DEXECUTORCH_ENABLE_LOGGING=ON \
   -DEXECUTORCH_BUILD_XNNPACK=ON \
   -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
   -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
@@ -21,7 +22,6 @@ cmake . -DCMAKE_INSTALL_PREFIX="${CMAKE_OUT}" \
   -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
   -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
   -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
-  -DCMAKE_BUILD_TYPE=Release \
   -B"${CMAKE_OUT}"
 
 if [ "$(uname)" == "Darwin" ]; then
@@ -29,20 +29,20 @@ if [ "$(uname)" == "Darwin" ]; then
 else
   CMAKE_JOBS=$(( $(nproc) - 1 ))
 fi
-cmake --build "${CMAKE_OUT}" -j "${CMAKE_JOBS}" --target install --config Release
+cmake --build "${CMAKE_OUT}" -j "${CMAKE_JOBS}" --target install
 
 cmake extension/android \
   -DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK}/build/cmake/android.toolchain.cmake \
   -DANDROID_ABI="${ANDROID_ABI}" \
   -DANDROID_PLATFORM=android-23 \
   -DCMAKE_INSTALL_PREFIX="${CMAKE_OUT}" \
+  -DEXECUTORCH_ENABLE_LOGGING=ON \
   -DEXECUTORCH_BUILD_EXTENSION_RUNNER_UTIL=ON \
   -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
   -DEXECUTORCH_BUILD_LLAMA_JNI=ON \
-  -DCMAKE_BUILD_TYPE=Release \
   -B"${CMAKE_OUT}"/extension/android
 
-cmake --build "${CMAKE_OUT}"/extension/android -j "${CMAKE_JOBS}" --config Release
+cmake --build "${CMAKE_OUT}"/extension/android -j "${CMAKE_JOBS}"
 
 BUILD_AAR_DIR="$(mktemp -d)"
 mkdir -p "${BUILD_AAR_DIR}/jni/${ANDROID_ABI}" "${BUILD_AAR_DIR}/libs"

i then rebuilt the app, and tried loading llama 3 8B again. goal: see if memory runs out, according to executorch logs

export ANDROID_NDK="/Users/william/Library/Android/sdk/ndk/25.2.9519653"
export ANDROID_ABI="arm64-v8a"
export ANDROID_HOME="/Users/william/Library/Android/sdk"
export EXECUTORCH_USE_TIKTOKEN=ON
pushd extension/android
./gradlew build
popd
pushd examples/demo-apps/android/LlamaDemo
./gradlew :app:setup
popd
pushd examples/demo-apps/android/LlamaDemo
./gradlew :app:installDebug
popd

interestingly, it no longer crashes. rather, when i enter a prompt like "cheese", it responds with an empty string.
logcat_out_cheese.txt

screenshot: https://github.com/user-attachments/assets/66ef6138-d217-4197-b542-d3f600ba8ef5

here is the logcat output from the seconds before and after entering the prompt "cheese"

there appears to be some issue with loading the tokenizer. i built this with export EXECUTORCH_USE_TIKTOKEN=ON, so it should use the tiktoken tokenizer. again, this is llama 3 8B... which i think is with tiktoken.

relatedly however, i see that my settings for "model" and "tokenizer" are still from when i ran llama 2 7B. i have no way of clearing the "tokenizer" setting.

maybe this is causing a problem? do i need to also push tokenizer.model to the device storage, and select it alongside the model?

screenshot: https://github.com/user-attachments/assets/814a5b46-0e70-4050-a7b5-215ddf2c2ccd

kirklandsign · 2024-09-27T16:22:17Z

Thanks for the update @grisaitis

yes you need to push the tokenizer file and select it. Please make sure to use llama 3 tokenizer (tiktoken) for llama 3 model

grisaitis · 2024-10-02T18:05:49Z

Thanks so much for your help! Sorry for not following up sooner.

I indeed got the demo to work with LLaMA 3.2 3B. It works beautifully.

My only feedback is to modify the documentation so that, also in the Android section (or perhaps before anything platform-specific?), it is mentioned to rename tokenizer.model to tokenizer.bin. i missed this commentary (until command-F'ing "tokenizer") because i wasn't building for iOS.

kirklandsign · 2024-10-02T18:15:54Z

My only feedback is to modify the documentation so that, also in the Android section (or perhaps before anything platform-specific?), it is mentioned to rename tokenizer.model to tokenizer.bin. i missed this commentary (until command-F'ing "tokenizer") because i wasn't building for iOS.

In the latest main it should work with both names (see

executorch/examples/demo-apps/android/LlamaDemo/app/src/main/java/com/example/executorchllamademo/SettingsActivity.java

Lines 345 to 346 in c48d867

    
           String[] binFiles = listLocalFile("/data/local/tmp/llama/", ".bin"); 
        
           String[] modelFiles = listLocalFile("/data/local/tmp/llama/", ".model");

)

dvorjackz added bug Something isn't working Android Android building and execution related. labels Sep 24, 2024

Olivia-liu assigned kirklandsign Sep 24, 2024

Olivia-liu added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 24, 2024

kirklandsign closed this as completed Oct 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v0.4.0-rc2 Android Demo App for Llama crashing on Pixel 8 when first prompt submitted #5582

v0.4.0-rc2 Android Demo App for Llama crashing on Pixel 8 when first prompt submitted #5582

grisaitis commented Sep 24, 2024 •

edited

Loading

dvorjackz commented Sep 24, 2024

kirklandsign commented Sep 24, 2024

grisaitis commented Sep 24, 2024

kirklandsign commented Sep 24, 2024

grisaitis commented Sep 25, 2024 •

edited

Loading

grisaitis commented Sep 25, 2024 •

edited

Loading

kirklandsign commented Sep 25, 2024

kirklandsign commented Sep 25, 2024

grisaitis commented Sep 26, 2024

kirklandsign commented Sep 26, 2024

grisaitis commented Sep 27, 2024 •

edited

Loading

kirklandsign commented Sep 27, 2024 •

edited

Loading

grisaitis commented Oct 2, 2024 •

edited

Loading

kirklandsign commented Oct 2, 2024

v0.4.0-rc2 Android Demo App for Llama crashing on Pixel 8 when first prompt submitted #5582

v0.4.0-rc2 Android Demo App for Llama crashing on Pixel 8 when first prompt submitted #5582

Comments

grisaitis commented Sep 24, 2024 • edited Loading

🐛 Describe the bug

Versions

build settings for the app

dvorjackz commented Sep 24, 2024

kirklandsign commented Sep 24, 2024

grisaitis commented Sep 24, 2024

kirklandsign commented Sep 24, 2024

grisaitis commented Sep 25, 2024 • edited Loading

grisaitis commented Sep 25, 2024 • edited Loading

kirklandsign commented Sep 25, 2024

kirklandsign commented Sep 25, 2024

grisaitis commented Sep 26, 2024

kirklandsign commented Sep 26, 2024

grisaitis commented Sep 27, 2024 • edited Loading

kirklandsign commented Sep 27, 2024 • edited Loading

grisaitis commented Oct 2, 2024 • edited Loading

kirklandsign commented Oct 2, 2024

grisaitis commented Sep 24, 2024 •

edited

Loading

grisaitis commented Sep 25, 2024 •

edited

Loading

grisaitis commented Sep 25, 2024 •

edited

Loading

grisaitis commented Sep 27, 2024 •

edited

Loading

kirklandsign commented Sep 27, 2024 •

edited

Loading

grisaitis commented Oct 2, 2024 •

edited

Loading