ztest: hash lock, keystore lock inversion #7112
Labels
Component: Test Suite
Indicates an issue with the test framework or a test case
Comments
behlendorf
added
the
Component: Test Suite
behlendorf
added a commit
to behlendorf/zfs
that referenced
this issue
Feb 1, 2018
A deadlock is possible between the hash_lock and keystore lock since they may not be taken in the same order. For this to occur two blocks need to hash to the same bucket. Full backtraces showing the deadlock are available in issue openzfs#7112. Resolve the issue by using mutex_tryenter() in arc_buf_access() which was introduced in 0873bb6. The reasoning being that a hash collision here is unlikely and updating the access info is not required for correctness. Since a collision is also unlikely it's not expected to have any performance impact. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> TEST_ZTEST_TIMEOUT=3600 Issue openzfs#7112
13 tasks
behlendorf
added a commit
to behlendorf/zfs
that referenced
this issue
Feb 1, 2018
A deadlock is possible between the hash_lock and keystore lock since they may not be taken in the same order. For this to occur two blocks need to hash to the same bucket. Full backtraces showing the deadlock are available in issue openzfs#7112. Resolve the issue by using mutex_tryenter() in arc_buf_access() which was introduced in 0873bb6. The reasoning being that a hash collision here is unlikely and updating the access info is not required for correctness. Since a collision is also unlikely it's not expected to have any performance impact. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> TEST_ZTEST_TIMEOUT=3600 Issue openzfs#7112
behlendorf
added a commit
to behlendorf/zfs
that referenced
this issue
Feb 2, 2018
The keystore.sk_dk_lock should not be held while performing I/O. Drop the lock when reading from disk and update the code so they the first successful caller adds the lock. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> TEST_ZTEST_TIMEOUT=3600 Issue openzfs#7112
13 tasks
behlendorf
added a commit
to behlendorf/zfs
that referenced
this issue
Feb 2, 2018
The keystore.sk_dk_lock should not be held while performing I/O. Drop the lock when reading from disk and update the code so they the first successful caller adds the lock. Improve error handling in spa_keystore_create_mapping_impl(). Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> TEST_ZTEST_TIMEOUT=3600 Issue openzfs#7112
Nasf-Fan
pushed a commit
to Nasf-Fan/zfs
that referenced
this issue
Feb 13, 2018
The keystore.sk_dk_lock should not be held while performing I/O. Drop the lock when reading from disk and update the code so they the first successful caller adds the key. Improve error handling in spa_keystore_create_mapping_impl(). Reviewed by: Thomas Caputi <tcaputi@datto.com> Reviewed-by: RageLtMan <rageltman@sempervictus> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#7112 Closes openzfs#7115
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
System information
Describe the problem you're observing
Lock inversion introduced by 0873bb6 and encryption.
Describe how to reproduce the problem
Occasionally observed when running
ztest, thus far never during normal usage.The text was updated successfully, but these errors were encountered: