Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Monitor: fix metric timespan to restore Storage Account PT1H metrics #40367

Merged

Conversation

zmoog
Copy link
Contributor

@zmoog zmoog commented Jul 26, 2024

Proposed commit message

Move the timespan logic into a dedicated buildTimespan() function with a test for each supported use case.

Some Azure services have longer latency between service usage and metric availability. For example, the Storage Account capacity metrics (Blob capacity, etc.) have a PT1H time grain and become available after one hour. Service X also has PT1H metrics, however become available after a few minutes.

This PR restores the core of the older timespan logic the Azure Monitor metricset was using before the regression introduced by the PR #36823.

However, the buildTimespan() does not restore the interval * (-2) part because doubling the interval causes duplicates.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • Test with Microsoft.ContainerRegistry/registries metrics (PT1M and PT1H)
  • Test with Microsoft.ContainerInstance/containerGroups metrics (PT5M)
  • Test with Microsoft.KeyVault/vaults metrics (PT1M)
  • Test with Storage Account metrics (PT5M and PT1H)
  • Test with Virtual Machines metrics (PT5M)

Related issues

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 26, 2024
@zmoog zmoog self-assigned this Jul 26, 2024
@zmoog zmoog added Team:obs-ds-hosted-services Label for the Observability Hosted Services team bug and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 26, 2024
Copy link
Contributor

mergify bot commented Jul 26, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @zmoog? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@zmoog zmoog changed the title Azure Monitor: Update metric timespan to support PT1H metrics in the Storage Account Azure Monitor: Update metric timespan to support Storage Account PT1H metrics Jul 26, 2024
@zmoog zmoog changed the title Azure Monitor: Update metric timespan to support Storage Account PT1H metrics Azure Monitor: fix metric timespan to restore Storage Account PT1H metrics Jul 29, 2024
@zmoog zmoog force-pushed the zmoog/azure-monitor-fix-storage-account-pt1h-metrics branch from f86e730 to cda2f82 Compare July 30, 2024 16:42
Add one test for each expected use case.
@zmoog zmoog marked this pull request as ready for review July 30, 2024 22:20
@zmoog zmoog requested a review from a team as a code owner July 30, 2024 22:20
@elasticmachine
Copy link
Collaborator

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

@zmoog zmoog added backport-8.15 Automated backport to the 8.15 branch with mergify backport-v8.14.0 Automated backport with mergify labels Jul 30, 2024
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@zmoog zmoog merged commit 5fccb0d into elastic:main Aug 1, 2024
19 checks passed
@zmoog zmoog deleted the zmoog/azure-monitor-fix-storage-account-pt1h-metrics branch August 1, 2024 10:42
mergify bot pushed a commit that referenced this pull request Aug 1, 2024
…trics (#40367)

Move the timespan logic into a dedicated `buildTimespan()` function with a test for each supported use case.

Some Azure services have longer latency between service usage and metric availability. For example, the Storage Account capacity metrics (Blob capacity, etc.) have a PT1H time grain and become available after one hour. Service X also has PT1H metrics, however become available after a few minutes.

This PR restores the core of the [older timespan logic](https://github.com/elastic/beats/blob/d3facc808d2ba293a42b2ad3fc8e21b66c5f2a7f/x-pack/metricbeat/module/azure/client.go#L110-L116) the Azure Monitor metricset was using before the regression introduced by the PR #36823.

However, the `buildTimespan()` does not restore the `interval * (-2)` part because doubling the interval causes duplicates.

(cherry picked from commit 5fccb0d)
mergify bot pushed a commit that referenced this pull request Aug 1, 2024
…trics (#40367)

Move the timespan logic into a dedicated `buildTimespan()` function with a test for each supported use case.

Some Azure services have longer latency between service usage and metric availability. For example, the Storage Account capacity metrics (Blob capacity, etc.) have a PT1H time grain and become available after one hour. Service X also has PT1H metrics, however become available after a few minutes.

This PR restores the core of the [older timespan logic](https://github.com/elastic/beats/blob/d3facc808d2ba293a42b2ad3fc8e21b66c5f2a7f/x-pack/metricbeat/module/azure/client.go#L110-L116) the Azure Monitor metricset was using before the regression introduced by the PR #36823.

However, the `buildTimespan()` does not restore the `interval * (-2)` part because doubling the interval causes duplicates.

(cherry picked from commit 5fccb0d)
zmoog added a commit that referenced this pull request Aug 1, 2024
…trics (#40367) (#40414)

Move the timespan logic into a dedicated `buildTimespan()` function with a test for each supported use case.

Some Azure services have longer latency between service usage and metric availability. For example, the Storage Account capacity metrics (Blob capacity, etc.) have a PT1H time grain and become available after one hour. Service X also has PT1H metrics, however become available after a few minutes.

This PR restores the core of the [older timespan logic](https://github.com/elastic/beats/blob/d3facc808d2ba293a42b2ad3fc8e21b66c5f2a7f/x-pack/metricbeat/module/azure/client.go#L110-L116) the Azure Monitor metricset was using before the regression introduced by the PR #36823.

However, the `buildTimespan()` does not restore the `interval * (-2)` part because doubling the interval causes duplicates.

(cherry picked from commit 5fccb0d)

Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co>
zmoog added a commit that referenced this pull request Aug 1, 2024
…trics (#40367) (#40413)

Move the timespan logic into a dedicated `buildTimespan()` function with a test for each supported use case.

Some Azure services have longer latency between service usage and metric availability. For example, the Storage Account capacity metrics (Blob capacity, etc.) have a PT1H time grain and become available after one hour. Service X also has PT1H metrics, however become available after a few minutes.

This PR restores the core of the [older timespan logic](https://github.com/elastic/beats/blob/d3facc808d2ba293a42b2ad3fc8e21b66c5f2a7f/x-pack/metricbeat/module/azure/client.go#L110-L116) the Azure Monitor metricset was using before the regression introduced by the PR #36823.

However, the `buildTimespan()` does not restore the `interval * (-2)` part because doubling the interval causes duplicates.

(cherry picked from commit 5fccb0d)

Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.15 Automated backport to the 8.15 branch with mergify backport-v8.14.0 Automated backport with mergify bug Team:obs-ds-hosted-services Label for the Observability Hosted Services team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Azure Monitor stops collecting PT1H metrics for Storage Account
3 participants