Job hangs after unknown crash

[safa007]

We recently started experiencing this issue where we see a crash at the end of the Terminate emulator step, and after this step the job just hangs infinitely until it's cancelled or times out. The logs aren't really helpful, we just see a UniversalExceptionRaise: (os/kern) failure (5) at the very end of the job. It does seem like the AVD creation is successful however.

I tried resetting the machine this is happening on, and after 2 successful runs we started seeing the issue again. I've tested on v2.25.0 and v2.28.0 and the result is the same on both versions.

Has anyone experienced this recently?

EDIT: After setting force-avd-creation to false we are still getting the UniversalExceptionRaise: (os/kern) failure (5) error however the job does not hang and it correctly moves on to the next step. So it seems this is only happening when a new AVD is being created. We set force-avd-creation to true on every test run in order to prevent out of memory issues on the emulator.

EDIT2: Still seeing the issue with force-avd-creation to false as well.

Generated AVD snapshot
::group::Terminate Emulator
Terminate Emulator
  /Users/administrator/.android/sdk/platform-tools/adb -s emulator-5554 emu kill
  saving arm snapshot.... !!!
  
  OK: killing emulator, bye bye
  OK
  INFO    | Wait for emulator (pid 10852) 20 seconds to shutdown gracefully before kill;you can set environment variable ANDROID_EMULATOR_WAIT_TIME_BEFORE_KILL(in seconds) to change the default value (20 seconds)
  ::endgroup::
saving done.... !!!

[10854:273912:20230616,105830.126506:WARNING crash_report_exception_handler.cc:235] UniversalExceptionRaise: (os/kern) failure (5)

Here's our GHA setup

      - name: Create AVD
        uses: reactivecircus/android-emulator-runner@v2.25.0
        with:
          api-level: 29
          arch: arm64-v8a
          profile: pixel_5
          avd-name: macOS-avd-arm64-v8a-29
          disk-size: 16G
          force-avd-creation: true
          emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
          channel: canary
          disable-animations: false
          script: echo "Generated AVD snapshot"

[safa007]

Solved.

For those of you that are struggling with this as well the issue turned out to be the emulator version. One of the latest releases 32.1.12 has a known issue which has been fixed in the latest emulator release. Make sure you are adding target: google_apis to receive the latest emulator version.

[carlospontual]

@safa007 how did you fix it? I've changed my target to google_apis but I'm still getting my jobs hanging without being able to kill the emulator...

here's my config:

      with:
        api-level: 30
        arch: arm64-v8a
        channel: stable
        target: google_apis
        profile: pixel_6
        avd-name: Test_Pixel6_Google_fhd_2
        force-avd-creation: false
        emulator-options: -no-window -gpu host -noaudio -no-boot-anim -camera-back none
        disable-animations: true

PS: if the AVD is re-created, I get the same exception you listed above (UniversalExceptionRaise: (os/kern) failure (5)) but the job doesn't hang...

Thanks!

[safa007]

@carlospontual Make sure to clear your runner's avd, gradle, and .android cache so that a new run can try to download the most up-to-date emulator version. I would double check that the runner is not using that faulty emulator version by checking the logs.

[carlospontual]

@safa007

I don't think I'm using the faulty version, what do you say?

  Starting emulator.
  /bin/sh -c \/Users/ec2-user/Library/Android/sdk/emulator/emulator -avd Test_Pixel6_Google_fhd_2 -no-window -gpu host -noaudio -no-boot-anim -camera-back none &
  INFO    | Android emulator version 33.1.16.0 (build_id 10511200) (CL:N/A)
  INFO    | Found systemPath /Users/ec2-user/Library/Android/sdk/system-images/android-30/google_apis/arm64-v8a/

[safa007]

@carlospontual could possibly be a different issue then 🤔

[JuscaMolaiwa]

@safa007 @carlospontual @hostilefork @rfc2822
Hey, I am experiencing the same issue here, been trying with different system images however seems like the arch type arm64-v8a is the issue and I am using macos.

So I have created an emulator on the macos agent then when launching it, it crashes See the details:

4_Install AndroidSDK ⚙️.txt

5_Set AVD&Emulator📱.txt

2024-02-20T10:55:14.2113280Z Available Android Virtual Devices:
2024-02-20T10:55:14.2131940Z Name: Pixel_6_Pro
2024-02-20T10:55:14.3135310Z Device: pixel_6_pro (Google)
2024-02-20T10:55:14.3427850Z Path: /Users/runner/.android/avd/Pixel_6_Pro.avd
2024-02-20T10:55:14.3430280Z Target: Default Android System Image
2024-02-20T10:55:14.3483170Z Based on: Android 10.0 (Q) Tag/ABI: default/arm64-v8a
2024-02-20T10:55:14.3516580Z Sdcard: 512 MB
2024-02-20T10:55:14.3602800Z ----------------------------------------------------------------------------------------------------------------------
2024-02-20T10:55:14.3676970Z Emulator Path: /Users/runner/Library/Android/sdk/tools/emulator
2024-02-20T10:55:14.3717080Z Launching the Pixel_6_Pro emulator
2024-02-20T10:55:14.3797630Z ----------------------------------------------------------------------------------------------------------------------
2024-02-20T10:55:14.5618950Z * daemon not running; starting now at tcp:5037
2024-02-20T10:55:15.2649830Z WARNING | Please update the emulator to one that supports the feature(s): Vulkan
2024-02-20T10:55:15.6677300Z WARNING | FeatureControl is requesting a non existing feature.
2024-02-20T10:55:17.2164280Z I0220 10:55:17.216054 a682600 HealthMonitor.cpp:279] HealthMonitor disabled.
2024-02-20T10:55:17.3113070Z I0220 10:55:17.310862 a682600 FrameBuffer.cpp:486] Graphics Adapter Vendor Google (Apple Inc.)
2024-02-20T10:55:17.4116260Z I0220 10:55:17.310895 a682600 FrameBuffer.cpp:487] Graphics Adapter Android Emulator OpenGL ES Translator (Apple Software Renderer)
2024-02-20T10:55:17.4513170Z I0220 10:55:17.310901 a682600 FrameBuffer.cpp:488] Graphics API Version OpenGL ES 2.0 (2.1 APPLE-19.5.1)
2024-02-20T10:55:17.4540410Z I0220 10:55:17.310905 a682600 FrameBuffer.cpp:489] Graphics API Extensions GL_OES_EGL_sync GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_depth24 GL_OES_depth32 GL_OES_element_index_uint GL_OES_texture_float GL_OES_texture_float_linear GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth_texture GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_packed_depth_stencil GL_OES_vertex_half_float GL_OES_texture_npot GL_EXT_color_buffer_float GL_EXT_texture_format_BGRA8888 GL_APPLE_texture_format_BGRA8888
2024-02-20T10:55:17.4559250Z I0220 10:55:17.310910 a682600 FrameBuffer.cpp:490] Graphics Device Extensions N/A
2024-02-20T10:55:17.5630420Z INFO | Storing crashdata in: , detection is enabled for process: 6198
2024-02-20T10:55:17.6491130Z INFO | Duplicate loglines will be removed, if you wish to see each individual line launch with the -log-nofilter flag.
2024-02-20T10:55:17.6716180Z INFO | Warning: QMetaObject::connectSlotsByName: No matching signal for on_new_posture_requested(int) ((null):0, (null))
2024-02-20T10:55:17.7032030Z INFO | Warning: QMetaObject::connectSlotsByName: No matching signal for on_dismiss_posture_selection_dialog() ((null):0, (null))
2024-02-20T10:55:17.8035040Z INFO | Warning: QMetaObject::connectSlotsByName: No matching signal for on_new_resizable_requested(PresetEmulatorSizeType) ((null):0, (null))
2024-02-20T10:55:17.8105150Z INFO | Warning: QMetaObject::connectSlotsByName: No matching signal for on_dismiss_resizable_dialog() ((null):0, (null))
2024-02-20T10:55:17.8150860Z library_mode host gpu mode host
2024-02-20T10:55:17.8249410Z INFO | Warning: QCssParser::parseHexColor: Unknown color name '#FFFFFFs' ((null):0, (null))
2024-02-20T10:55:17.8337220Z WARNING | /etc/localtime does not point to zoneinfo-compatible timezone name
2024-02-20T10:55:17.8416630Z WARNING | The emulator now requires a signed jwt token for gRPC access! Use the -grpc flag if you really want an open unprotected grpc port
2024-02-20T10:55:17.8447530Z INFO | Using security allow list from: /Users/runner/Library/Android/sdk/emulator/lib/emulator_access.json
2024-02-20T10:55:17.8516430Z WARNING | *** Basic token auth should only be used by android-studio ***
2024-02-20T10:55:17.8538790Z INFO | The active JSON Web Key Sets can be found here: /Users/runner/Library/Caches/TemporaryItems/avd/running/6198/jwks/10a58059-8f35-47da-b127-93c04c2b0459/active.jwk
2024-02-20T10:55:17.8554360Z INFO | Scanning /Users/runner/Library/Caches/TemporaryItems/avd/running/6198/jwks/10a58059-8f35-47da-b127-93c04c2b0459 for jwk keys.
2024-02-20T10:55:17.8591860Z INFO | Started GRPC server at 127.0.0.1:8554, security: Local, auth: +token
2024-02-20T10:55:17.8624890Z INFO | Advertising in: /Users/runner/Library/Caches/TemporaryItems/avd/running/pid_6198.ini
2024-02-20T10:55:17.8648390Z * daemon started successfully
2024-02-20T10:55:17.8681170Z Emulator is not yet booted. Please wait...
2024-02-20T10:55:17.8712590Z List of devices attached
2024-02-20T10:55:17.8788800Z emulator-5554 offline
2024-02-20T10:55:17.8844580Z
2024-02-20T10:55:17.9259390Z [6204:28880:20240220,105517.925555:WARNING crash_report_exception_handler.cc:235] UniversalExceptionRaise: (os/kern) failure (5)
2024-02-20T10:55:22.7547960Z /Users/runner/work/1/s/pipelines/setup/android/macos/SettingAVD&Emulator.sh: line 60: 6198 Segmentation fault: 11 "$emulator_executable" -avd "Pixel_6_Pro" -gpu host -no-audio -netfast -no-snapshot