vdpa/virtio: replace doorbell relay with notifier #52
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| DRV_LOG(ERR, "%s virtq doorbell relay failed ret:%d", | ||
| priv->vdev->device->name, ret); | ||
| return ret; | ||
| if (priv->configured){ |
There was a problem hiding this comment.
better add a space between ) and {
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| priv->vdev->device->name, ret); | ||
| return ret; | ||
| if (priv->configured){ | ||
| virtio_vdpa_lcore_id = rte_get_next_lcore(virtio_vdpa_lcore_id, 1, 1); |
There was a problem hiding this comment.
We get this lcore_id using rte_get_next_lcore every time. So do we need to define it as a global var? maybe temp var?
There was a problem hiding this comment.
we use global to prevent use same core next time
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| if (priv->configured){ | ||
| virtio_vdpa_lcore_id = rte_get_next_lcore(virtio_vdpa_lcore_id, 1, 1); | ||
| notify_work = rte_zmalloc(NULL, sizeof(*notify_work), 0); | ||
| if(!notify_work) { |
There was a problem hiding this comment.
better add a space after if
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| virtio_vdpa_lcore_id = rte_get_next_lcore(virtio_vdpa_lcore_id, 1, 1); | ||
| notify_work = rte_zmalloc(NULL, sizeof(*notify_work), 0); | ||
| if(!notify_work) { | ||
| DRV_LOG(ERR, "%s vfid %d failed request memory", priv->vdev->device->name, priv->vf_id); |
There was a problem hiding this comment.
Maybe "failed to alloc notify work"?
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| @@ -964,6 +829,24 @@ virtio_vdpa_dev_close_work(void *arg) | |||
| priv->dev_work_flag = VIRTIO_VDPA_DEV_CLOSE_WORK_DONE; | |||
| return ret; | |||
| } | |||
| static int | |||
There was a problem hiding this comment.
maybe add a blank line before this
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| virtio_vdpa_lcore_id = rte_get_next_lcore(virtio_vdpa_lcore_id, 1, 1); | ||
| notify_work = rte_zmalloc(NULL, sizeof(*notify_work), 0); | ||
| if(!notify_work) { | ||
| DRV_LOG(ERR, "%s vfid %d failed request memory", vdev->device->name, priv->vf_id); |
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| @@ -1233,6 +1117,23 @@ virtio_vdpa_dev_config(int vid) | |||
| } | |||
| priv->lm_status = VIRTIO_S_RUNNING; | |||
|
|
|||
| virtio_vdpa_lcore_id = rte_get_next_lcore(virtio_vdpa_lcore_id, 1, 1); | |||
| notify_work = rte_zmalloc(NULL, sizeof(*notify_work), 0); | |||
| if(!notify_work) { | |||
Ch3n60x
reviewed
Dec 19, 2023
drivers/vdpa/virtio/virtio_vdpa.c
Outdated
| DRV_LOG(INFO, "%s vfid %d launch notifier work lcore:%d", vdev->device->name, priv->vf_id, virtio_vdpa_lcore_id); | ||
| ret = rte_eal_remote_launch(virtio_vdpa_dev_notifier_work, notify_work, virtio_vdpa_lcore_id); | ||
| if (ret) { | ||
| DRV_LOG(ERR, "%s vfid %d failed launch notifier work ret:%d lcore:%d", vdev->device->name, priv->vf_id, ret, virtio_vdpa_lcore_id); |
There was a problem hiding this comment.
line 1130 and 1133 seems too long. maybe using two lines?
|
Did some test. result shows before this commit, set_vring_state could take more than 0.01s for some queue. But after this change, all queues set_vring_state should be below 0.0001s. |
Call fd is -1 when pre-config, in order to prevent 2nd restore modify queue info which takes long time, modify queue's msix to default value. But dpdk case will different, 2nd restoer time will longger than kernel Signed-off-by: Kailiang <kailiangz@nvidia.com>
Doorbell relay is used before, but it is time consuming. Use notifier instead and use DPDK work to do notify because we can't wait when we process vhost msg, otherwise, deadlock will happend, so, use work thread to finish the job Signed-off-by: Kailiang <kailiangz@nvidia.com>
2308524
approved these changes
Jan 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Doorbell relay is used before, but it is time consuming. Use notifier instead and use DPDK work to do notify because we can't wait when we process vhost msg, otherwise, deadlock will happend, so, use work thread to finish the job