Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mantle/kola: add detection for a kernel soft lockup #3450

Merged
merged 1 commit into from
May 3, 2023

Commits on May 3, 2023

  1. mantle/kola: add detection for a kernel soft lockup

    Recently we saw a test with many soft lockup messages like:
    
    ```
    [ 4159.779792] watchdog: BUG: soft lockup - CPU#0 stuck for 883s! [kworker/u2:0:10]
    [ 4159.780488] Modules linked in:
    [ 4159.780787] CPU: 0 PID: 10 Comm: kworker/u2:0 Tainted: G             L    -------  ---  6.4.0-0.rc0.20230502git865fdb08197e.11.fc39.x86_64 #1
    [ 4159.780787] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-1.fc37 04/01/2014
    [ 4159.780787] Workqueue: ftrace_check_wq ftrace_check_work_func
    [ 4159.780787] RIP: 0010:get_symbol_pos+0x5d/0x140
    [ 4159.780787] Code: 63 0c 8d 08 d0 6e a0 85 c9 79 0a 48 f7 d1 48 03 0d 50 c2 5a 01 48 39 cf 48 0f 42 f0 48 0f 43 d0 48 89 f0 48 29 d0 48 83 f8 01 <77> ca 48 85 d2 75 08 eb 4b 48 83 ea 01 74 45 8d 42 ff 48 98 48 63
    [ 4159.780787] RSP: 0018:ffffb1970005bb80 EFLAGS: 00000202
    [ 4159.780787] RAX: 0000000000000032 RBX: ffffffff9f22edb4 RCX: ffffffff9f22ed00
    [ 4159.780787] RDX: 0000000000004436 RSI: 0000000000004468 RDI: ffffffff9f22edb4
    [ 4159.780787] RBP: ffffb1970005bbce R08: 0000000000000000 R09: ffffb1970005bbc0
    [ 4159.780787] R10: 0000000000000000 R11: 00000000000320a7 R12: 0000000000000000
    [ 4159.780787] R13: 0000000000000000 R14: ffffb1970005bbc0 R15: 0000000000000000
    [ 4159.780787] FS:  0000000000000000(0000) GS:ffff9fa07ec00000(0000) knlGS:0000000000000000
    [ 4159.780787] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [ 4159.780787] CR2: ffff9fa04fe01000 CR3: 000000000e022000 CR4: 0000000000350ef0
    [ 4159.780787] Call Trace:
    [ 4159.780787]  <TASK>
    [ 4159.780787]  kallsyms_lookup_buildid+0x4d/0x130
    [ 4159.780787]  test_for_valid_rec+0x64/0xb0
    [ 4159.780787]  ftrace_check_work_func+0x3b/0x60
    [ 4159.780787]  process_one_work+0x1c7/0x3d0
    [ 4159.780787]  worker_thread+0x51/0x390
    [ 4159.780787]  ? __pfx_worker_thread+0x10/0x10
    [ 4159.780787]  kthread+0xf7/0x130
    [ 4159.780787]  ? __pfx_kthread+0x10/0x10
    [ 4159.780787]  ret_from_fork+0x2c/0x50
    [ 4159.780787]  </TASK>
    ```
    
    Let's try to detect and report this.
    dustymabe committed May 3, 2023
    Configuration menu
    Copy the full SHA
    d0fc64f View commit details
    Browse the repository at this point in the history