ZVOL Panics / VERIFY(dr->dt.dl.dr_override_state != DR_IN_DMU_SYNC) failed #6238
Comments
|
I've been unable to re-produce the issue now for a few days. There were reports of a library causing kvm issues with a recent Arch update, and I had other seg-faults with the kvm process. I've switched to running a different distro now. If anything new appears, I'll open a new issue. |
This was referenced Aug 3, 2017
behlendorf
pushed a commit
that referenced
this issue
Aug 21, 2017
Since OpenZFS 7578 (1b7c1e5) if we have a ZVOL with logbias=throughput we will force WR_INDIRECT itxs in zvol_log_write() setting itx->itx_lr offset and length to the offset and length of the BIO from zvol_write()->zvol_log_write(): these offset and length are later used to take a range lock in zillog->zl_get_data function: zvol_get_data(). Now suppose we have a ZVOL with blocksize=8K and push 4K writes to offset 0: we will only be range-locking 0-4096. This means the ASSERTion we make in dbuf_unoverride() is no longer valid because now dmu_sync() is called from zilog's get_data functions holding a partial lock on the dbuf. Fix this by taking a range lock on the whole block in zvol_get_data(). Reviewed-by: Chunwei Chen <tuxoko@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Closes #6238 Closes #6315 Closes #6356 Closes #6477
tonyhutter
pushed a commit
that referenced
this issue
Aug 22, 2017
Since OpenZFS 7578 (1b7c1e5) if we have a ZVOL with logbias=throughput we will force WR_INDIRECT itxs in zvol_log_write() setting itx->itx_lr offset and length to the offset and length of the BIO from zvol_write()->zvol_log_write(): these offset and length are later used to take a range lock in zillog->zl_get_data function: zvol_get_data(). Now suppose we have a ZVOL with blocksize=8K and push 4K writes to offset 0: we will only be range-locking 0-4096. This means the ASSERTion we make in dbuf_unoverride() is no longer valid because now dmu_sync() is called from zilog's get_data functions holding a partial lock on the dbuf. Fix this by taking a range lock on the whole block in zvol_get_data(). Reviewed-by: Chunwei Chen <tuxoko@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Closes #6238 Closes #6315 Closes #6356 Closes #6477
SidBB
pushed a commit
to catalogicsoftware/zfs
that referenced
this issue
Aug 31, 2017
Since OpenZFS 7578 (1b7c1e5) if we have a ZVOL with logbias=throughput we will force WR_INDIRECT itxs in zvol_log_write() setting itx->itx_lr offset and length to the offset and length of the BIO from zvol_write()->zvol_log_write(): these offset and length are later used to take a range lock in zillog->zl_get_data function: zvol_get_data(). Now suppose we have a ZVOL with blocksize=8K and push 4K writes to offset 0: we will only be range-locking 0-4096. This means the ASSERTion we make in dbuf_unoverride() is no longer valid because now dmu_sync() is called from zilog's get_data functions holding a partial lock on the dbuf. Fix this by taking a range lock on the whole block in zvol_get_data(). Reviewed-by: Chunwei Chen <tuxoko@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Closes openzfs#6238 Closes openzfs#6315 Closes openzfs#6356 Closes openzfs#6477
Fabian-Gruenbichler
pushed a commit
to Fabian-Gruenbichler/zfs
that referenced
this issue
Sep 29, 2017
Since OpenZFS 7578 (1b7c1e5) if we have a ZVOL with logbias=throughput we will force WR_INDIRECT itxs in zvol_log_write() setting itx->itx_lr offset and length to the offset and length of the BIO from zvol_write()->zvol_log_write(): these offset and length are later used to take a range lock in zillog->zl_get_data function: zvol_get_data(). Now suppose we have a ZVOL with blocksize=8K and push 4K writes to offset 0: we will only be range-locking 0-4096. This means the ASSERTion we make in dbuf_unoverride() is no longer valid because now dmu_sync() is called from zilog's get_data functions holding a partial lock on the dbuf. Fix this by taking a range lock on the whole block in zvol_get_data(). Reviewed-by: Chunwei Chen <tuxoko@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: loli10K <ezomori.nozomu@gmail.com> Closes openzfs#6238 Closes openzfs#6315 Closes openzfs#6356 Closes openzfs#6477
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've had recent,consistent zvol panics. They seem to happen regularly when I use kvm with the zvol block device. Mine is set up with no compression, kvm's Cache Mode = none, using virtio scsi. I also tried putting cache mode back to kvm's default (I think it uses writeback), but also saw the panic which make it seem o_direct is not an issue.
I started having issues with an existing zvol, so I created a new zvol and immediately see the same issue during an os installation. My previous zvol would constantly panic just running software inside the vm.
I scrubbed and ran "zdb -mc" on the pool with no issues reported.
I'm using latest git versions (as of today) of spl and zfs. I'm running with 4.9.32 kernel on Arch Linux.
My steps to re-produce (using a clean zvol):
(Update Find) It appears that if I don't use -b 128K and go with the default, then there are no panics. As soon as I re-create the test zvol with the -b 128K, the panics come back. I tested on two separate pools.
I've been getting regular panics, so I've been running a debug spl/zfs. Hopefully, this ASSERT message and stacks that appear will help:
And here's another stack of a prior panic with some more cores:
The text was updated successfully, but these errors were encountered: