Slave Usage Monitoring UI #1405

matush-v · 2017-01-24T19:41:47Z

This is the front-end for the slave usage tracking. Definitely a work in progress, but the general structure is there. Feel free to edit the task lists.

Design Tasks:

slave button styling
slave stat menu styling
aggregate page styling: do we want aggregate info/what do we want here?

Functionality Tasks:

what are the thresholds and where are they defined
over subscription issues
timestamp display locale
clicking on a slave stat copies to clipboard
how to show dead slaves?
show/sort by rack info?
filtering/sorting options

Updated - 2/21/2017

cc: @ssalinas @wsorenson @tpetr
BE: #1400

ssalinas · 2017-01-24T19:48:32Z

A few comments based on your task list above:

There is a method in utils for converting those timestamps to a more human-readble format
Not sure what you categorize as disabled slaves, but it might be good to separate out at least the 'Dead' ones like on the slaves page
Would be interesting to be able to aggregate by rack (i.e. one big box of summed up metrics for each rack)
would be nice to be able to search/filter for a slave in some way. Not sure what the best way for that would be since we don't want those boxes to be text-heavy at all
For the over-subscription piece, we should add extra attributes that Singularity will look for on the slave. If those attributes are present, it will use that as the 'total' for mem/cpu/etc. If they are not present it will use the values reported by the slave. This will need to be a change on the backend PR

Will look more at the actual js and leave comments there shortly

matush-v · 2017-01-24T19:50:19Z

The actual js has a lot of room for improvement. Modularization is one I'm working on now. I think that review is better saved for once the design is decided since it can change a lot before that.

wsorenson · 2017-01-25T21:46:16Z

cool!

We need to merge the data with the slave APIs so that they should hostname and cpusUsed vs cpusAllocated (as opposed to the raw no.)

Maybe that was obvious / not implemented yet, wanted to make sure!

ssalinas · 2017-01-25T21:49:23Z

Yep we talked about using the attribute values (falling back to the resource ones) as the denominator for getting the usage percentages already

And yes @darcatron , would be nice if we sub that slave ID in with the hostname, would be much easier to reason about/read in the ui that way

ssalinas

What you have here will function. But, in terms of having cleaner, easier to uderstand code in react. I think it might be better to separate these out into more components.

Right now you have the one large SlaveUsage componenet. I think you could easily have a component for each Slave box (that can take props like the warn/crit levels and actual numerical stats), and maybe even a sub component for the info box that has the raw stats displayed

ssalinas · 2017-01-25T21:50:51Z

SingularityUI/app/actions/api/slaves.es6

@@ -2,14 +2,14 @@ import { buildApiAction, buildJsonApiAction } from './base';

 export const FetchSlaves = buildApiAction(
  'FETCH_SLAVES',
-  {url: '/slaves'}


why did you remove all of the leading slashes here?

the buildApiAction already has /api/ so it's just an extra slash. ../api//slaves will work, but it's not proper so I figured i'd remove it. I can drop it from the rest of the files too, but prob better for another pr

turns out this is handled in staging so the buildApiAction doesn't return the extra slash

ssalinas · 2017-01-25T21:54:16Z

SingularityUI/app/components/machines/SlaveUsage.jsx

+    var warningStyle = 'warning';
+    var okStyle = 'success';
+
+    if (isStatCritical(slave, props.cpusUsedStat) || isStatCritical(slave, props.memoryBytesUsedStat) || isStatCritical(slave, props.numTasksStat)) {


we should probably catch the exceptions you are throwing in the case where we can't properly find the stats. It'd be better to show less stats (excluding the ones that threw an exception), rather than throwing js errors all the way up

ssalinas · 2017-01-25T21:55:27Z

SingularityUI/app/components/machines/SlaveUsage.jsx

+  };
+
+  const slaveWithStats = (slave, index, bsStyle, glyphicon) => (
+    <Dropdown key={slave.slaveId} id={index.toString()}>


Any reasoning behind using dropdown here? Seems like an odd component choice

The dropdown is the little stat menu you get when clicking on a slave btn. Using Dropdown over DropdownMenu allows more styling flexibility. Not sure if that's what you were talking about

ssalinas · 2017-02-02T15:18:44Z

SingularityUI/app/rootComponent.jsx

@@ -36,6 +36,7 @@ const rootComponent = (Wrapped, title, refresh = _.noop, refreshInterval = true,
      }).catch((reason) => {
        // Boot React errors out of the promise so they can be picked up by Sentry
        setTimeout(() => {
+          console.log(reason.stack);


don't forget to remove these after you're done testing ;)

I'm never done testing 😈

matush-v · 2017-02-22T18:09:51Z

@ssalinas got the updates in for the default values. Once you think it's correct for staging, i'll set our config for shortening the slave name to true and deploy it

ssalinas · 2017-03-01T14:34:39Z

SingularityService/src/main/java/com/hubspot/singularity/views/IndexView.java

@@ -239,6 +239,10 @@ public String getRedirectOnUnauthorizedUrl() {
    return redirectOnUnauthorizedUrl;
  }

+  public boolean isShortenSlaveUsageHostname() {


FYI @darcatron , without the getter here, this value is not available when rendering the template, so {{ shortenSlaveUsageHostname }} would come back empty in the rendered template

demobox · 2017-03-17T18:47:55Z

Just related to this, or perhaps for potential inspiration: we are currently doing similar monitoring of our Mesos agent usage and utilization via https://github.com/Capgemini/mesos-ui (video at https://youtu.be/bKbOod8Pn4E)

basic render of slave usage page

edc4824

matush-v added 4 commits January 24, 2017 15:28

code cleanup

edb96da

wired up slave usages to api

0dff51f

humans read this

6b2876c

check for real resource values, easily copy stats

60245af

ssalinas reviewed Jan 25, 2017

View reviewed changes

add css

5f08796

matush-v force-pushed the slave-usage branch from 40813ee to 5f08796 Compare January 25, 2017 22:03

matush-v added 6 commits January 25, 2017 17:53

show host name instead of slave id

96337da

better string formatting

0ffb5d5

more updates

3c4bf7a

better slave usage components

16dccca

add constants file

c731720

further component breakdown

6ef209c

matush-v force-pushed the slave-usage branch from 3b4a977 to 6ef209c Compare February 2, 2017 02:18

matush-v added the hs_staging label Feb 2, 2017

update packages

e8dd569

matush-v force-pushed the slave-usage branch from 5b6e119 to e8dd569 Compare February 2, 2017 02:42

ssalinas reviewed Feb 2, 2017

View reviewed changes

matush-v added 4 commits February 2, 2017 11:16

fix props

e8c30dd

fix to work in staging

e40ee63

dont need no loggin

edf3048

extra api '/' is fixed in staging :|

261f9bf

matush-v added the hs_qa label Feb 2, 2017

ssalinas changed the title ~~[WIP] Slave Usage Monitoring~~ Slave Usage Monitoring UI Feb 2, 2017

ssalinas added this to the 0.14.0 milestone Feb 9, 2017

add thresholds to config

6a3c9b6

matush-v added 11 commits February 22, 2017 13:31

ref error

83286ae

only check active slaves

c903b6d

slave usage reports are only for active slaves

09cc66f

clarity

1b4d25e

branches hate me

bd3f960

add links to tasks for slave

13ed92b

grandient style heatmap for mem and cpu utilization

2850ee0

better clipboard copying

734a636

remove unused config and styles

f84d57c

sort slave heat map, adjust gradient, refactor

515141f

extend range to account for hundredths place

a8dec94

ssalinas added the hs_stable label Feb 27, 2017

add missing getter

27388de

ssalinas reviewed Mar 1, 2017

View reviewed changes

mouseover callouts, style changes

8a366de

ssalinas modified the milestones: 0.14.0, 0.15.0 Mar 13, 2017

increase margin, tooltip on bottom

c9fe927

ssalinas added 2 commits March 20, 2017 10:04

fix merge conflicts with master

08445bb

missed merges in package.json

a37cdb0

ssalinas merged commit e2d5c5a into master Mar 20, 2017

ssalinas deleted the slave-usage branch March 20, 2017 14:57

ssalinas removed hs_qa labels Mar 20, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Slave Usage Monitoring UI #1405

Slave Usage Monitoring UI #1405

matush-v commented Jan 24, 2017 •

edited

Loading

ssalinas commented Jan 24, 2017

matush-v commented Jan 24, 2017

wsorenson commented Jan 25, 2017

ssalinas commented Jan 25, 2017

ssalinas left a comment

ssalinas Jan 25, 2017

matush-v Jan 25, 2017 •

edited

Loading

matush-v Feb 2, 2017

ssalinas Jan 25, 2017

ssalinas Jan 25, 2017

matush-v Jan 25, 2017 •

edited

Loading

ssalinas Feb 2, 2017

matush-v Feb 2, 2017

matush-v commented Feb 22, 2017

ssalinas Mar 1, 2017

demobox commented Mar 17, 2017 •

edited

Loading

Slave Usage Monitoring UI #1405

Slave Usage Monitoring UI #1405

Conversation

matush-v commented Jan 24, 2017 • edited Loading

ssalinas commented Jan 24, 2017

matush-v commented Jan 24, 2017

wsorenson commented Jan 25, 2017

ssalinas commented Jan 25, 2017

ssalinas left a comment

Choose a reason for hiding this comment

ssalinas Jan 25, 2017

Choose a reason for hiding this comment

matush-v Jan 25, 2017 • edited Loading

Choose a reason for hiding this comment

matush-v Feb 2, 2017

Choose a reason for hiding this comment

ssalinas Jan 25, 2017

Choose a reason for hiding this comment

ssalinas Jan 25, 2017

Choose a reason for hiding this comment

matush-v Jan 25, 2017 • edited Loading

Choose a reason for hiding this comment

ssalinas Feb 2, 2017

Choose a reason for hiding this comment

matush-v Feb 2, 2017

Choose a reason for hiding this comment

matush-v commented Feb 22, 2017

ssalinas Mar 1, 2017

Choose a reason for hiding this comment

demobox commented Mar 17, 2017 • edited Loading

matush-v commented Jan 24, 2017 •

edited

Loading

matush-v Jan 25, 2017 •

edited

Loading

matush-v Jan 25, 2017 •

edited

Loading

demobox commented Mar 17, 2017 •

edited

Loading