Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add request type to telemetry #287

Merged
merged 10 commits into from
May 23, 2023
Merged

Add request type to telemetry #287

merged 10 commits into from
May 23, 2023

Conversation

TingDaoK
Copy link
Contributor

@TingDaoK TingDaoK commented May 12, 2023

follow up PR for #271 (comment)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@TingDaoK TingDaoK marked this pull request as ready for review May 12, 2023 17:51
@codecov-commenter
Copy link

codecov-commenter commented May 12, 2023

Codecov Report

Merging #287 (a2a0159) into main (4e9c731) will increase coverage by 0.08%.
The diff coverage is 87.50%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #287      +/-   ##
==========================================
+ Coverage   88.22%   88.30%   +0.08%     
==========================================
  Files          16       16              
  Lines        4406     4431      +25     
==========================================
+ Hits         3887     3913      +26     
+ Misses        519      518       -1     
Impacted Files Coverage Δ
source/s3_auto_ranged_get.c 98.53% <66.66%> (-0.72%) ⬇️
source/s3_auto_ranged_put.c 92.68% <100.00%> (-0.15%) ⬇️
source/s3_request.c 95.19% <100.00%> (+0.17%) ⬆️

... and 1 file with indirect coverage changes

@TingDaoK TingDaoK mentioned this pull request May 15, 2023
Copy link
Member

@jamesbornholt jamesbornholt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like what we need, thanks!

include/aws/s3/s3_client.h Outdated Show resolved Hide resolved
include/aws/s3/s3_client.h Show resolved Hide resolved
enum aws_s3_request_type {

/* Auto Ranged PUT request types */
AWS_S3_REQUEST_TYPE_AUTO_RANGED_PUT_LIST_PARTS,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe simply use the S3 API name.

I'm not sure it's a great idea to use our private subclass names as part of the public API. Frees us up to change the names of things, or move to some other design scheme (break up subclasses differently, change to some other design pattern, etc)

Suggested change
AWS_S3_REQUEST_TYPE_AUTO_RANGED_PUT_LIST_PARTS,
AWS_S3_REQUEST_TYPE_LIST_PARTS,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure how this data is going to be used.
Would it be better to just report a string in the metrics, instead of an enum?
Would it be OK if the string changed in some future version?
Is it better that we indicate what's going on under the hood, which is liable to change. Or is it better to be stable in what we report, and keep out details that might change in the future?

I don't know. maybe talk with someone who's thinking about using this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure as well. I did this mainly because:

  • I think we already expose the metrics as some inspect of the internal implementation. So, those private objects are as well part of the inspect of how we actually doing the job under the hood
  • We are not matching the S3 api one by one. Like get the object size from copy, get object part, which is a ranged get, etc?

BUT, I think it makes more sense to keep public stuff align with the public doc from S3, and user knows the meta request were made. So, it's okay if copy/put shared the same API calls. And, keep us free to change in the future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The S3 API name is all we really want to know. We already know what type of meta request it is from the aws_s3_meta_request pointer the telemetry callback gets; we just don't have a way to distinguish e.g. HeadObject and GetObject calls in auto_ranged_get (so we can report on their latency separately).

include/aws/s3/s3_client.h Outdated Show resolved Hide resolved
@TingDaoK TingDaoK merged commit 7fb4474 into main May 23, 2023
@TingDaoK TingDaoK deleted the add-request-type branch May 23, 2023 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants