Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prometheus view to ad hoc metrics #1677

Merged
merged 3 commits into from
Aug 25, 2017

Conversation

yaacov
Copy link
Member

@yaacov yaacov commented Jul 12, 2017

Description

Add Prometheus as an external data source

Screenshot
gifrecord_2017-08-25_182715

@yaacov
Copy link
Member Author

yaacov commented Jul 12, 2017

@miq-bot add_label compute/containers

wip because it requires: ManageIQ/manageiq-providers-kubernetes#71

@simon3z @cben @moolitayer @nimrodshn please review

@miq-bot
Copy link
Member

miq-bot commented Aug 21, 2017

This pull request is not mergeable. Please rebase and repush.

@yaacov yaacov force-pushed the add-prometheus-ad-hoc-metrics branch from e6e88a3 to 1fc036f Compare August 21, 2017 06:42
@yaacov yaacov changed the title [WIP] Add prometheus view to ad hoc metrics Add prometheus view to ad hoc metrics Aug 24, 2017
@yaacov
Copy link
Member Author

yaacov commented Aug 24, 2017

@miq-bot miq-bot removed the wip label Aug 24, 2017
@yaacov yaacov force-pushed the add-prometheus-ad-hoc-metrics branch from 1fc036f to 32fc71f Compare August 24, 2017 08:54
@yaacov
Copy link
Member Author

yaacov commented Aug 24, 2017

cc @ilackarms this PR is updated

@yaacov
Copy link
Member Author

yaacov commented Aug 24, 2017

@joelddiaz this is our take on Prometheus metrics view in Manageiq, please review
(the infrastructure PR this one was dependent on, is merged, so this one is ready for review)

cc @simon3z

@ilackarms
Copy link
Contributor

tested working. LGTM!

@@ -78,7 +82,7 @@ def data(query)
rescue StandardError => e
{
:parameters => params,
:error => ActionView::Base.full_sanitizer.sanitize(e.to_s) + " " + _("(Please check your Hawkular server)")
:error => ActionView::Base.full_sanitizer.sanitize(e.to_s) + " " + _("(Please check your #{@db_name} server)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't work, you need to _("(Please check your %{type} server)") % {:type => @db_name}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thanks, I do not think i18n ... , copied your fix :-)

Copy link
Contributor

@himdel himdel Aug 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:) The only thing to keep it in mind really is that the _ function can only translate it if it finds an exact match in its db. So any interpolation has to be done on the result of _, not before.

jobs.map { |id| { :label => labelize(id), :value => id } }
end

def add_last_readings(metrics)
Copy link
Contributor

@himdel himdel Aug 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead method? (Or just useless? :))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 done, moved implementation to parent class

"default" => "Default",
"admin" => "Admin",
"openshift-infra" => "OpenShift Infra"
"_system" => "System",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any chance these should be translated?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thanks makes seance.

@himdel
Copy link
Contributor

himdel commented Aug 25, 2017

@yaacov Trying to test in ui, but the current set-miq-providers fails with

/home/himdel/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:50:in `refresh': HTTP status code 401, 401 Unauthorized (EmsRefresh::Refreshers::EmsRefresherMixin::PartialRefreshError)
	from /home/himdel/manageiq/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh'
	from /home/himdel/manageiq/app/models/ems_refresh.rb:97:in `block in refresh'
	from /home/himdel/manageiq/app/models/ems_refresh.rb:96:in `each'
	from /home/himdel/manageiq/app/models/ems_refresh.rb:96:in `refresh'
	from set-miq-providers.rb:56:in `create_prov_with_auth'
	from set-miq-providers.rb:77:in `<main>'

Can you take a look please?

From evm.log:

[----] W, [2017-08-25T14:17:44.066245 #24502:2ac4c19d50d4]  WARN -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager#authentication_check_no_validation) type: [:bearer] for [10000000000030] [EngLab] Validation failed: error, HTTP status code 401, 401 Unauthorized
[----] W, [2017-08-25T14:17:44.066406 #24502:2ac4c19d50d4]  WARN -- : MIQ(AuthToken#validation_failed) [ExtManagementSystem] [10000000000030], previously valid on: , previous status: []
[----] I, [2017-08-25T14:17:44.080154 #24502:2ac4c19d50d4]  INFO -- : MIQ(MiqQueue.put) Message id: [10000004399921],  id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openshift::ContainerManager", 10000000000030], "ems_auth_error", {}]
[----] W, [2017-08-25T14:17:44.453801 #24502:2ac4c19d50d4]  WARN -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager#authentication_check_no_validation) type: [:hawkular] for [10000000000030] [EngLab] Validation failed: error, <html>
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1">

  <style type="text/css">
  /*!
   * Bootstrap v3.3.5 (http://getbootstrap.com)
   * Copyright 2011-2015 Tw...
[----] W, [2017-08-25T14:17:44.454079 #24502:2ac4c19d50d4]  WARN -- : MIQ(AuthToken#validation_failed) [ExtManagementSystem] [10000000000030], previously valid on: , previous status: []
[----] I, [2017-08-25T14:17:44.468833 #24502:2ac4c19d50d4]  INFO -- : MIQ(MiqQueue.put) Message id: [10000004399922],  id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openshift::ContainerManager", 10000000000030], "ems_auth_error", {}]
[----] I, [2017-08-25T14:17:44.471526 #24502:2ac4c19d50d4]  INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) Refreshing all targets...
[----] I, [2017-08-25T14:17:44.471579 #24502:2ac4c19d50d4]  INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) EMS: [EngLab], id: [10000000000030] Refreshing targets for EMS...
[----] I, [2017-08-25T14:17:44.471612 #24502:2ac4c19d50d4]  INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) EMS: [EngLab], id: [10000000000030]   ManageIQ::Providers::Openshift::ContainerManager [EngLab] id [10000000000030]
[----] I, [2017-08-25T14:17:44.471694 #24502:2ac4c19d50d4]  INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager#with_provider_connection) Connecting through ManageIQ::Providers::Openshift::ContainerManager: [EngLab]
[----] E, [2017-08-25T14:17:44.842675 #24502:2ac4c19d50d4] ERROR -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) EMS: [EngLab], id: [10000000000030] Refresh failed
[----] E, [2017-08-25T14:17:44.842862 #24502:2ac4c19d50d4] ERROR -- : [KubeException]: 401 Unauthorized  Method:[block in method_missing]
[----] E, [2017-08-25T14:17:44.842935 #24502:2ac4c19d50d4] ERROR -- : /home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/kubeclient-2.4.0/lib/kubeclient/common.rb:117:in `rescue in handle_exception'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/kubeclient-2.4.0/lib/kubeclient/common.rb:109:in `handle_exception'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/kubeclient-2.4.0/lib/kubeclient/common.rb:461:in `fetch_entities'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/kubeclient-2.4.0/lib/kubeclient/common.rb:451:in `load_entities'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/kubeclient-2.4.0/lib/kubeclient/common.rb:121:in `discover'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/kubeclient-2.4.0/lib/kubeclient/common.rb:88:in `method_missing'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-kubernetes-bba2b66404e4/app/models/manageiq/providers/kubernetes/container_manager/refresher_mixin.rb:16:in `block in fetch_entities'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-kubernetes-bba2b66404e4/app/models/manageiq/providers/kubernetes/container_manager/refresher_mixin.rb:14:in `each'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-kubernetes-bba2b66404e4/app/models/manageiq/providers/kubernetes/container_manager/refresher_mixin.rb:14:in `each_with_object'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-kubernetes-bba2b66404e4/app/models/manageiq/providers/kubernetes/container_manager/refresher_mixin.rb:14:in `fetch_entities'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-openshift-f35ccc218169/app/models/manageiq/providers/openshift/container_manager/refresher.rb:32:in `block in collect_inventory_for_targets'
/home/himdel/manageiq/app/models/ext_management_system.rb:382:in `with_provider_connection'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-providers-openshift-f35ccc218169/app/models/manageiq/providers/openshift/container_manager/refresher.rb:31:in `collect_inventory_for_targets'
/home/himdel/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:80:in `block in refresh_targets_for_ems'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-29bc8de18faf/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-29bc8de18faf/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/home/himdel/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:79:in `refresh_targets_for_ems'
/home/himdel/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block (2 levels) in refresh'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-29bc8de18faf/lib/gems/pending/util/extensions/miq-benchmark.rb:11:in `realtime_store'
/home/himdel/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/bundler/gems/manageiq-gems-pending-29bc8de18faf/lib/gems/pending/util/extensions/miq-benchmark.rb:30:in `realtime_block'
/home/himdel/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:24:in `block in refresh'
/home/himdel/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `each'
/home/himdel/manageiq/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:14:in `refresh'
/home/himdel/manageiq/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh'
/home/himdel/manageiq/app/models/ems_refresh.rb:97:in `block in refresh'
/home/himdel/manageiq/app/models/ems_refresh.rb:96:in `each'
/home/himdel/manageiq/app/models/ems_refresh.rb:96:in `refresh'
set-miq-providers.rb:56:in `create_prov_with_auth'
set-miq-providers.rb:77:in `<main>'

(And the old provider showed a long message about it not being available, but I can't get to it again, sorry.)

@yaacov
Copy link
Member Author

yaacov commented Aug 25, 2017

Can you take a look please?

sorry, all my machines are down/broken because for bad openshit 3.7 images ..., re-spinning 3.6 images now, will ping you when they will be running.

@yaacov
Copy link
Member Author

yaacov commented Aug 25, 2017

@himdel ^^

@yaacov yaacov force-pushed the add-prometheus-ad-hoc-metrics branch from 32fc71f to ef8a881 Compare August 25, 2017 15:04
@yaacov
Copy link
Member Author

yaacov commented Aug 25, 2017

first machine is up (with prometheus metrics) working on the first next one.
set providers is updated:
https://github.com/yaacov/miq-scripts/blob/master/set-miq-providers.rb

@miq-bot
Copy link
Member

miq-bot commented Aug 25, 2017

Checked commits yaacov/manageiq-ui-classic@6db8abe~...b1ea37f with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
3 files checked, 0 offenses detected
Everything looks fine. 👍

@yaacov
Copy link
Member Author

yaacov commented Aug 25, 2017

@himdel what to do about codeclimate ?

@himdel
Copy link
Contributor

himdel commented Aug 25, 2017

@yaacov No need to do anything, the threshold is set too low for ruby (18), we were discussing increasing the threshold today ..

Cc @martinpovolny: mass 25 is definitely too low :)

ASSIGN_TOS[:chargeback_compute] = {
    "enterprise"             => N_("The Enterprise"),
    "ext_management_system"  => N_("Selected Providers"),
    "ems_cluster"            => N_("Selected Cluster / Deployment Roles"),
    "ems_container"          => N_("Selected Containers Providers"),

and

  TENANT_LABEL_SPECIAL_CASES = {
    "_system"                      => _("System"),
    "_ops"                         => _("Operations"),
    "default"                      => _("Default"),
    "admin"                        => _("Admin"),

are definitely not similar enough :).

@himdel himdel self-assigned this Aug 25, 2017
@himdel himdel merged commit 2054002 into ManageIQ:master Aug 25, 2017
@himdel
Copy link
Contributor

himdel commented Aug 25, 2017

Tested in the UI, LGTM :)

@himdel himdel added this to the Sprint 68 Ending Sep 4, 2017 milestone Aug 25, 2017
ZitaNemeckova added a commit to ZitaNemeckova/manageiq-ui-classic that referenced this pull request Apr 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants