Skip to content

Commit

Permalink
Move gethrtime() calls out of vdev queue lock
Browse files Browse the repository at this point in the history
This dramatically reduces the lock contention on systems with slower
(non-TSC) timecounters.  With TSC the difference is minimal, but since
this lock is pretty congested, any improvement counts.  Plus I don't
see any reason to do it under the lock other than the latency of the
lock itself, which this change actually reduces.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #12281
  • Loading branch information
amotin authored Jul 6, 2021
1 parent 6e4e3c3 commit 97752ba
Showing 1 changed file with 5 additions and 6 deletions.
11 changes: 5 additions & 6 deletions module/zfs/vdev_queue.c
Original file line number Diff line number Diff line change
Expand Up @@ -912,9 +912,9 @@ vdev_queue_io(zio_t *zio)
}

zio->io_flags |= ZIO_FLAG_DONT_CACHE | ZIO_FLAG_DONT_QUEUE;
zio->io_timestamp = gethrtime();

mutex_enter(&vq->vq_lock);
zio->io_timestamp = gethrtime();
vdev_queue_io_add(vq, zio);
nio = vdev_queue_io_to_issue(vq);
mutex_exit(&vq->vq_lock);
Expand All @@ -936,14 +936,13 @@ vdev_queue_io_done(zio_t *zio)
vdev_queue_t *vq = &zio->io_vd->vdev_queue;
zio_t *nio;

mutex_enter(&vq->vq_lock);
hrtime_t now = gethrtime();
vq->vq_io_complete_ts = now;
vq->vq_io_delta_ts = zio->io_delta = now - zio->io_timestamp;

mutex_enter(&vq->vq_lock);
vdev_queue_pending_remove(vq, zio);

zio->io_delta = gethrtime() - zio->io_timestamp;
vq->vq_io_complete_ts = gethrtime();
vq->vq_io_delta_ts = vq->vq_io_complete_ts - zio->io_timestamp;

while ((nio = vdev_queue_io_to_issue(vq)) != NULL) {
mutex_exit(&vq->vq_lock);
if (nio->io_done == vdev_queue_agg_io_done) {
Expand Down

0 comments on commit 97752ba

Please sign in to comment.