Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.7] backport IO metrics from main #4490

Merged

Conversation

kalyazin
Copy link
Contributor

@kalyazin kalyazin commented Mar 8, 2024

Changes

Backporting extra IO metrics (#4486) from main.

Reason

We would like for them to appear in prod as soon as possible.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • [ ] If a specific issue led to this PR, this PR closes the issue.
  • The description of changes is clear and encompassing.
  • Any required documentation changes (code and docs) are included in this
    PR.
  • [ ] API changes follow the Runbook for Firecracker API changes.
  • User-facing changes are mentioned in CHANGELOG.md.
  • [ ] All added/changed functionality is tested.
  • [ ] New TODOs link to an issue.
  • Commits meet
    contribution quality standards.

  • This functionality cannot be added in rust-vmm.

This is needed to collect metrics depending on
the number of remaining requests in the queue.

(cherry picked from commit d480849)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
This is for it to be available in block and net modules.

(cherry picked from commit 430a034)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
LatencyAggregateMetrics metrics are dictionaries
rather than ints, so they require special treatment.

(cherry picked from commit 012a38a)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
These metrics measure how long Firecracker VMM thread
is blocked on the read/write syscalls when accessing
data in the drive file.
By looking at them, we will be able to see what portion
of block latency is attributed to factors external
to Firecracker.

(cherry picked from commit e00d62d)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
This metric accumulates the number of pending
requests sitting in the virtio queue while a request
is being processed.
By looking at this number, one may make a decision
on whether an async IO solution would be beneficial.

(cherry picked from commit 58a87ed)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
This metric measures how long Firecracker VMM thread
is blocked on the write syscalls when accessing a tap device.
By looking at it, we will be able to see what portion
of tx net latency is attributed to factors external
to Firecracker.

(cherry picked from commit f7d8a33)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
This metric accumulates the number of pending
requests sitting in the Tx virtio queue while a request
is being processed.
This may give an insight at how sensible it would
be to think of processing ingress traffic parallelly.

(cherry picked from commit 3753f35)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
Copy link

codecov bot commented Mar 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.65%. Comparing base (1e8f903) to head (afb0c25).

Additional details and impacted files
@@                 Coverage Diff                  @@
##           firecracker-v1.7    #4490      +/-   ##
====================================================
+ Coverage             81.63%   81.65%   +0.01%     
====================================================
  Files                   243      243              
  Lines                 29803    29834      +31     
====================================================
+ Hits                  24329    24360      +31     
  Misses                 5474     5474              
Flag Coverage Δ
4.14-c5n.metal 78.97% <100.00%> (+0.02%) ⬆️
4.14-c7g.metal ?
4.14-m5d.metal ?
4.14-m5n.metal 78.96% <100.00%> (+0.02%) ⬆️
4.14-m6a.metal 78.10% <100.00%> (+0.02%) ⬆️
4.14-m6g.metal 77.06% <100.00%> (+0.02%) ⬆️
4.14-m6i.metal 78.95% <100.00%> (+0.02%) ⬆️
4.14-m7g.metal 77.06% <100.00%> (+0.02%) ⬆️
5.10-c5n.metal 81.62% <100.00%> (+0.02%) ⬆️
5.10-c7g.metal ?
5.10-m5d.metal ?
5.10-m5n.metal 81.61% <100.00%> (+0.02%) ⬆️
5.10-m6a.metal 80.85% <100.00%> (+0.02%) ⬆️
5.10-m6g.metal 79.94% <100.00%> (+0.02%) ⬆️
5.10-m6i.metal 81.61% <100.00%> (+0.02%) ⬆️
5.10-m7g.metal 79.94% <100.00%> (+0.02%) ⬆️
6.1-c5n.metal 81.62% <100.00%> (+0.02%) ⬆️
6.1-c7g.metal ?
6.1-m5d.metal ?
6.1-m5n.metal 81.61% <100.00%> (+0.02%) ⬆️
6.1-m6a.metal 80.85% <100.00%> (+0.02%) ⬆️
6.1-m6g.metal 79.94% <100.00%> (+0.02%) ⬆️
6.1-m6i.metal 81.61% <100.00%> (+0.02%) ⬆️
6.1-m7g.metal 79.94% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kalyazin kalyazin marked this pull request as ready for review March 8, 2024 16:24
@kalyazin kalyazin self-assigned this Mar 8, 2024
@kalyazin kalyazin added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Mar 8, 2024
ShadowCurse
ShadowCurse previously approved these changes Mar 8, 2024
roypat
roypat previously approved these changes Mar 8, 2024
wearyzen
wearyzen previously approved these changes Mar 8, 2024
@wearyzen wearyzen self-requested a review March 8, 2024 16:32
CHANGELOG.md Outdated Show resolved Hide resolved
 - block/net file/tap access latencies
 - block/net virtio queue backlog lengths

(cherry picked from commit 448df60)

Signed-off-by: Nikita Kalyazin <kalyazin@amazon.com>
Copy link
Contributor

@wearyzen wearyzen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this!

@kalyazin kalyazin merged commit 36184b4 into firecracker-microvm:firecracker-v1.7 Mar 8, 2024
7 checks passed
@kalyazin kalyazin deleted the io_metrics_1.7 branch March 8, 2024 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Awaiting review Indicates that a pull request is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants