Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] Add googlecloud compute overview dashboard #16819

Merged
merged 13 commits into from
Mar 18, 2020
Merged

[Metricbeat] Add googlecloud compute overview dashboard #16819

merged 13 commits into from
Mar 18, 2020

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Mar 4, 2020

This PR is to add googlecloud compute overview dashboard:
Screen Shot 2020-03-17 at 10 33 08 AM (2)

How to test it

  1. enable googlecloud module
./metricbeat modules enable googlecloud
  1. make sure in the google cloud account you are using for testing, there are VMs running. Note on what are the regions they belong to and remember to change region in step 3 configuration.

  2. change modules.d/googlecloud.yml to:

- module: googlecloud
  metricsets:
    - compute
  region: "us-central1"
  project_id: elastic-observability
  credentials_file_path: "/Users/kaiyansheng/Downloads/elastic-observability-d17781618202.json"
  exclude_labels: false
  period: 300s
- module: googlecloud
  metricsets:
    - compute
  region: "europe-west1"
  project_id: elastic-observability
  credentials_file_path: "/Users/kaiyansheng/Downloads/elastic-observability-d17781618202.json"
  exclude_labels: false
  period: 300s
  1. Start metricbeat and you should see data points in the compute dashboard shortly.

closes #16534

@kaiyan-sheng kaiyan-sheng requested a review from a team as a code owner March 4, 2020 21:04
@kaiyan-sheng kaiyan-sheng self-assigned this Mar 4, 2020
@kaiyan-sheng kaiyan-sheng added Metricbeat Metricbeat needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team labels Mar 4, 2020
CHANGELOG.next.asciidoc Outdated Show resolved Hide resolved
Copy link
Contributor

@sorantis sorantis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The filters can be combined into one control instead of having 3 separate ones.
  • instance.uptime can be added to the dashboard as a label between the filters and CPU graph.

@kaiyan-sheng
Copy link
Contributor Author

kaiyan-sheng commented Mar 6, 2020

  • The filters can be combined into one control instead of having 3 separate ones.

Good point, done!

  • instance.uptime can be added to the dashboard as a label between the filters and CPU graph.

Thanks for your suggestions @sorantis ! I just added uptime as a separate visualization. The problem is instance/uptime metric from GCP is delta kind, which means it measures the change in a value over a sample period. So with this uptime visualization, you won't be able to see numbers going up to larger than 60.

@sorantis
Copy link
Contributor

sorantis commented Mar 9, 2020

The problem is instance/uptime metric from GCP is delta kind, which means it measures the change in a value over a sample period. So with this uptime visualization, you won't be able to see numbers going up to larger than 60.

@kaiyan-sheng I see the issue. I was thinking about it as a True/False indicator(sorry for not making it clear), e.g. <green>Running 34<green>, <red>Down 19</red>. If there's no other way of interpreting this metric, then we should probably not show it at all.

@kaiyan-sheng
Copy link
Contributor Author

@sorantis Thank you for the suggestion. I changed uptime to show 100% Running 34 and less than 100% Running 2. With how googlecloud metrics are collected, this is still not accurate . Because when you set period=5m in gcp compute metricset, you get metrics every minute still for 3 times in each 5 minute.
Screen Shot 2020-03-09 at 1 12 13 PM
I will put this PR into work in progress till I figure out how to accurately display this uptime. Thanks!

@kaiyan-sheng kaiyan-sheng added the in progress Pull request is currently in progress. label Mar 9, 2020
@kaiyan-sheng
Copy link
Contributor Author

@sorantis What do you think about changing uptime using a gauge to show percentage here? I'd like to show True/False indicator but the data we are collecting here doesn't give me a good way of displaying the exact number of instances. I'm also working on figuring out how we can improve data collection shown as screenshot above. After figuring that out, I can come back and revisit the uptime visualization. Thanks!

@kaiyan-sheng kaiyan-sheng removed the in progress Pull request is currently in progress. label Mar 11, 2020
@sorantis
Copy link
Contributor

looks much better, @kaiyan-sheng, thanks for revisiting it!

@sorantis
Copy link
Contributor

nit: perhaps we can align all dropdowns vertically, and place the gauge between the filters and cpu utilization? this should save space, improve symmetry (and help my OCD :D)

@kaiyan-sheng
Copy link
Contributor Author

@sorantis Totally understand. I just made the change. Thanks!

Copy link
Contributor

@sorantis sorantis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kaiyan-sheng kaiyan-sheng merged commit 578e9b8 into elastic:master Mar 18, 2020
@kaiyan-sheng kaiyan-sheng deleted the compute_dashboard branch March 18, 2020 15:58
@kaiyan-sheng kaiyan-sheng added v7.7.0 and removed needs_backport PR is waiting to be backported to other branches. labels Mar 18, 2020
kaiyan-sheng added a commit that referenced this pull request Mar 19, 2020
…iew dashboard (#17093)

* [Metricbeat] Add googlecloud compute overview dashboard (#16819)

* Add googlecloud compute overview dashboard

(cherry picked from commit 578e9b8)
@kaiyan-sheng kaiyan-sheng added the test-plan Add this PR to be manual test plan label Mar 27, 2020
@andresrc andresrc added test-plan-added This PR has been added to the test plan and removed [zube]: Done labels Mar 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Metricbeat Metricbeat review Team:Platforms Label for the Integrations - Platforms team test-plan Add this PR to be manual test plan test-plan-added This PR has been added to the test plan v7.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Metricbeat] Add dashboard for compute metricset in googlecloud module
4 participants