Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] allow empty region/zone config param for storage metricset #17624

Merged
merged 3 commits into from
Apr 9, 2020
Merged

[Metricbeat] allow empty region/zone config param for storage metricset #17624

merged 3 commits into from
Apr 9, 2020

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Apr 8, 2020

What does this PR do?

This PR is to allow storage metricset with no region/zone config parameter because based on the documentation: If no region is specified, it will return metrics from all buckets.
https://github.com/elastic/beats/blob/master/x-pack/metricbeat/module/googlecloud/storage/_meta/docs.asciidoc

Why is it important?

GCP storage has two different classes, regional and multi-regional. When user wants to get storage metrics using Metricbeat for regional storage resources, a region/zone name can be specified in config. But if a region/zone is specified, no metrics can be queried from multi-regional storage resources.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

  1. Make sure your google cloud platform account has multiple storage accounts: both regional and multi-regional ones.
  2. Enable googlecloud module and modify googlecloud.yml to use:
- module: googlecloud
  metricsets:
    - storage
  project_id: elastic-observability
  credentials_file_path: "/Users/kaiyansheng/Downloads/elastic-observability-d17781618202.json"
  exclude_labels: false
  period: 300s
  1. Start metricbeat and you should see metrics collected from both multi-regional and regional storage resources.

Related issues

Closes #17623

Screenshots

Here is a screenshot for showing metrics collected from all regions with config:

- module: googlecloud
  metricsets:
    - storage
  project_id: elastic-observability
  credentials_file_path: "/Users/kaiyansheng/Downloads/elastic-observability-d17781618202.json"
  exclude_labels: false
  period: 300s

Screen Shot 2020-04-08 at 3 30 13 PM

@kaiyan-sheng kaiyan-sheng self-assigned this Apr 8, 2020
@kaiyan-sheng kaiyan-sheng added bug Metricbeat Metricbeat needs_backport PR is waiting to be backported to other branches. Team:Platforms Label for the Integrations - Platforms team labels Apr 8, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@kaiyan-sheng kaiyan-sheng requested a review from sayden April 8, 2020 21:34
Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@kaiyan-sheng kaiyan-sheng merged commit 3b7d29f into elastic:master Apr 9, 2020
@kaiyan-sheng kaiyan-sheng deleted the storage_gcp_doc branch April 9, 2020 15:05
@kaiyan-sheng kaiyan-sheng added v7.8.0 and removed needs_backport PR is waiting to be backported to other branches. labels Apr 9, 2020
syepes pushed a commit to syepes/beats that referenced this pull request Apr 9, 2020
…et (elastic#17624)

* allow empty region/zone config param for storage metricset
kaiyan-sheng added a commit that referenced this pull request Apr 9, 2020
…et (#17624) (#17643)

* allow empty region/zone config param for storage metricset

(cherry picked from commit 3b7d29f)
kaiyan-sheng added a commit that referenced this pull request Apr 9, 2020
…et (#17624) (#17642)

* allow empty region/zone config param for storage metricset

(cherry picked from commit 3b7d29f)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…et (elastic#17624) (elastic#17643)

* allow empty region/zone config param for storage metricset

(cherry picked from commit a8ef199)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Metricbeat Metricbeat Team:Platforms Label for the Integrations - Platforms team v7.7.0 v7.8.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Metricbeat GCP] Storage metricset requires region as input parameter
3 participants