Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cant mount previously working dataset - high CPU #7648

Closed
niksfirefly opened this issue Jun 20, 2018 · 1 comment
Closed

Cant mount previously working dataset - high CPU #7648

niksfirefly opened this issue Jun 20, 2018 · 1 comment
Labels
Status: Stale No recent activity for issue

Comments

@niksfirefly
Copy link

System information

Type Version/Name
Distribution Name Ubuntu
Distribution Version 18.04
Linux Kernel 4.15.0-23-generic #25-Ubuntu SMP Wed May 23 18:02:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ZFS Version 0.7.5-1ubuntu15
SPL Version 0.7.5-1ubuntu1

Problem

I cant mount previously working dataset.
I observe high CPU and kern.log errors instantly which leads host to become unresponsive.
How can i debug and restore data from this dataset?
Zpool scrub was done without errors.

How to reproduce the problem

zfs mount pool/lxd/containers/dev-firewalld
or
lxc start dev-firewalld

kern.log

kernel: [  787.291341] Call Trace:
kernel: [  787.291348]  dump_stack+0x63/0x8b
kernel: [  787.291355]  spl_dumpstack+0x42/0x50 [spl]
kernel: [  787.291360]  spl_panic+0xc8/0x110 [spl]
kernel: [  787.291364]  ? __update_load_avg_se.isra.38+0x1c0/0x1d0
kernel: [  787.291366]  ? __update_load_avg_se.isra.38+0x1c0/0x1d0
kernel: [  787.291368]  ? update_load_avg+0x57f/0x6e0
kernel: [  787.291371]  ? update_curr+0xf5/0x1d0
kernel: [  787.291373]  ? select_idle_sibling+0x383/0x410
kernel: [  787.291375]  ? __enqueue_entity+0x5c/0x60
kernel: [  787.291377]  ? enqueue_entity+0x10e/0x6b0
kernel: [  787.291379]  ? wakeup_preempt_entity.isra.69+0x41/0x50
kernel: [  787.291381]  ? wakeup_preempt_entity.isra.69+0x41/0x50
kernel: [  787.291436]  range_tree_add+0x90/0x300 [zfs]
kernel: [  787.291469]  metaslab_free_dva+0x12b/0x400 [zfs]
kernel: [  787.291499]  metaslab_free+0xdd/0xf0 [zfs]
kernel: [  787.291534]  zio_dva_free+0x1c/0x30 [zfs]
kernel: [  787.291567]  zio_wait+0xb2/0x1b0 [zfs]
kernel: [  787.291599]  zio_free+0x9a/0xe0 [zfs]
kernel: [  787.291628]  dsl_free+0x11/0x20 [zfs]
kernel: [  787.291652]  dsl_dataset_block_kill+0x27e/0x460 [zfs]
kernel: [  787.291673]  dbuf_write_done+0x182/0x1b0 [zfs]
kernel: [  787.291691]  arc_write_done+0x86/0x3f0 [zfs]
kernel: [  787.291723]  zio_done+0x449/0xdf0 [zfs]
kernel: [  787.291756]  zio_execute+0x90/0xf0 [zfs]
kernel: [  787.291762]  taskq_thread+0x2ab/0x4e0 [spl]
kernel: [  787.291764]  ? wake_up_q+0x80/0x80
kernel: [  787.291797]  ? zio_reexecute+0x390/0x390 [zfs]
kernel: [  787.291800]  kthread+0x121/0x140
kernel: [  787.291805]  ? taskq_thread_should_stop+0x70/0x70 [spl]
kernel: [  787.291808]  ? kthread_create_worker_on_cpu+0x70/0x70
kernel: [  787.291811]  ret_from_fork+0x35/0x40
kernel: [  967.742944] INFO: task z_wr_int_6:19627 blocked for more than 120 seconds.
kernel: [  967.742948]       Tainted: P           OE    4.15.0-23-generic #25-Ubuntu
kernel: [  967.742951] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kernel: [  967.742954] z_wr_int_6      D    0 19627      2 0x80000000
kernel: [  967.742958] Call Trace:
kernel: [  967.742965]  __schedule+0x297/0x8b0
kernel: [  967.742971]  ? select_idle_sibling+0x383/0x410
kernel: [  967.742977]  schedule+0x2c/0x80
kernel: [  967.742983]  schedule_preempt_disabled+0xe/0x10
kernel: [  967.742986]  __mutex_lock.isra.2+0x18c/0x4d0
kernel: [  967.742995]  ? spl_kmem_cache_alloc+0x64/0x140 [spl]
kernel: [  967.742999]  __mutex_lock_slowpath+0x13/0x20
kernel: [  967.743003]  ? __mutex_lock_slowpath+0x13/0x20
kernel: [  967.743006]  mutex_lock+0x2f/0x40
kernel: [  967.743072]  metaslab_free_dva+0xce/0x400 [zfs]
kernel: [  967.743138]  metaslab_free+0xdd/0xf0 [zfs]
kernel: [  967.743210]  zio_dva_free+0x1c/0x30 [zfs]
kernel: [  967.743278]  zio_wait+0xb2/0x1b0 [zfs]
kernel: [  967.743341]  zio_free+0x9a/0xe0 [zfs]
kernel: [  967.743405]  dsl_free+0x11/0x20 [zfs]
kernel: [  967.743463]  dsl_dataset_block_kill+0x27e/0x460 [zfs]
kernel: [  967.743509]  dbuf_write_done+0x182/0x1b0 [zfs]
kernel: [  967.743550]  arc_write_done+0x86/0x3f0 [zfs]
kernel: [  967.743619]  zio_done+0x449/0xdf0 [zfs]
kernel: [  967.743689]  zio_execute+0x90/0xf0 [zfs]
kernel: [  967.743699]  taskq_thread+0x2ab/0x4e0 [spl]
kernel: [  967.743704]  ? wake_up_q+0x80/0x80
kernel: [  967.743772]  ? zio_reexecute+0x390/0x390 [zfs]
kernel: [  967.743779]  kthread+0x121/0x140
kernel: [  967.743789]  ? taskq_thread_should_stop+0x70/0x70 [spl]
kernel: [  967.743795]  ? kthread_create_worker_on_cpu+0x70/0x70
kernel: [  967.743801]  ret_from_fork+0x35/0x40

@stale
Copy link

stale bot commented Aug 25, 2020

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Status: Stale No recent activity for issue label Aug 25, 2020
@stale stale bot closed this as completed Nov 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Stale No recent activity for issue
Projects
None yet
Development

No branches or pull requests

1 participant