UI: Topology Visualization #9077

DingoEatingFuzz · 2020-10-13T14:54:43Z

This adds a topology visualization to the UI. It can be used to the see the state of every client and allocation at a glance. You can read more about it on Discuss where prototypes were posted.

TODO

Integration tests for TopoViz
Integration tests for TopoViz::Datacenter
Acceptance tests for /topology
Merge api: add field filters to /v1/{allocations,nodes} #9055

github-actions · 2020-10-13T15:02:37Z

Ember Asset Size action

As of 8f94a98

Files that got Bigger 🚨:

File	raw	gzip
nomad-ui.js	+67.9 kB	+9.32 kB
vendor.js	+54.5 kB	+8.07 kB
nomad-ui.css	+3.78 kB	+693 B

Files that stayed the same size 🤷‍:

File	raw	gzip
vendor.css	0 B	0 B

github-actions · 2020-10-13T15:30:05Z

Ember Test Audit comparison

	master	`8f94a98`	change
passes	1393	1450	+57
failures	0	0	0
flaky	0	0	0
duration	6m 39s 156ms	8m 06s 150ms	+1m 26s 994ms

- Plot all datacenters - For each datacenter, plot all nodes - For each node, plot all allocations by memory and cpu - For empty nodes, highlight the emptiness - When hovering over allocations, give them visual focus

TBH, it's buggy and I don't like it.

…computing data

… structures Now all data loading happens in the TopoViz component as well as computation of resource proportions. Allocation selection state is also managed centrally uses a dedicated structure indexed by group key (job id and task group name). This way allocations don't need to be scanned at the node level, which is O(n) at the best (assuming no ember overhead on recomputes).

backspace

I documented some browser-specific problems but those can be addressed in followups, maybe the only thing I’d consider blocking is the changes to the environment file/default scenario, unless that was deliberate. I tried this out locally (only with Mirage) but it’s fun to toy with and seems like it’ll be useful!

backspace · 2020-10-15T13:55:18Z

ui/tests/unit/helpers/format-bytes-test.js

+
+  test('formats x > 1024 * 1024 * 1024 * 1024 as GiB, since it is the highest allowed unit', function(assert) {
+    assert.equal(formatBytes([1024 * 1024 * 1024 * 1024]), '1024 GiB');
+    assert.equal(formatBytes([1024 * 1024 * 1024 * 1024 * 4]), '4096 GiB');


nice, maybe I can use this elsewhere 😀

backspace · 2020-10-15T13:57:51Z

ui/tests/pages/components/topo-viz.js

+
+  allocationAssociationsArePresent: isPresent('[data-test-allocation-associations]'),
+  allocationAssociations: collection('[data-test-allocation-association]'),
+});


This is surely not something you have any influence on but starting with this file there’s no more syntax highlighting, strangely!

lol now this file has it but not the next 🙃

I find this happens pretty regularly with mega PRs.

backspace · 2020-10-15T14:04:01Z

ui/app/styles/components/legend.scss

+      margin-left: 1.5em;
+    }
+  }
+}


Maybe legend entries could be made non-wrapping somehow?

looks good 🎉

backspace · 2020-10-15T14:07:51Z

ui/app/styles/charts/topo-viz-node.scss

+    transform: translate(50%, 50%);
+    text-anchor: middle;
+    alignment-baseline: central;
+  }


Safari disappointments:

😱 Browsers, please.

whew, sad times but nice fix

backspace · 2020-10-15T14:12:07Z

ui/app/templates/components/topo-viz.hbs

+        {{/each}}
+      </g>
+    </svg>
+  {{/if}}


In Firefox on the Netlify deployment:

Strangely, resizing the window doesn’t change anything (I thought window-resize might be related) but zooming out a few steps caused it to become aligned properly and then stay that way when I returned to default zoom!

This is on Chrome locally with topoMedium scenario:

I’d think there shouldn’t be anything pointing to an empty client?

The resizing might be a bug, but the lines pointing to weird places is a quirk of the fixtures.

I didn't bother making sure allocations weren't "placed" on clients that didn't have room for them. The result is allocations taking up more than 100% of the width. So that line going to the empty client is actually going to the memory rect for the client in the left column, except it's beyond the width of the client.

It would be nice to make this better at some point, but I'm not sure how important it is since it's just for our local dev experience.

ah understood, doesn’t seem worth fixing to me then, thanks

backspace · 2020-10-15T15:45:20Z

ui/app/controllers/topology.js

+      .reduce((sum, memory) => sum + (memory || 0), 0);
+    const totalReservedCPU = node.allocations
+      .mapBy('cpu')
+      .reduce((sum, cpu) => sum + (cpu || 0), 0);


It looks like these reducer functions could be shared?

backspace · 2020-10-15T15:49:14Z

ui/app/components/topo-viz/node.js

+  // the yOffset to match heights with unstroked shapes.
+  get selectedYOffset() {
+    return this.height + 2.5;
+  }


I appreciate these explanations, sad that it has to be this way 😐

backspace · 2020-10-15T15:55:36Z

ui/app/components/topo-viz.js

+      // Steps are used to restrict the range of curves. The closer control points are placed, the less
+      // curvature the curve generator will generate.
+      const stepsMain = [0, 0.8, 1.0];
+      // The second prong the fork does not need to retrace the entire path from the activePoint


I appreciate these comments, would be nice if it were easy to have a reference diagram, but such is not the way hehe

backspace · 2020-10-15T15:57:04Z

ui/app/components/flex-masonry.js

+  @action
+  reflow() {
+    run.next(() => {
+      // There's nothing to do if this is  single column layout


backspace · 2020-10-15T15:57:37Z

ui/app/components/flex-masonry.js

+
+      // Set the max height of the container to the height of the tallest column
+      this.element.style.maxHeight = max(columns.mapBy('height')) + 1 + 'px';
+    });


I appreciate the explanations, I hope this ends up being useful somewhere else too!

backspace · 2020-10-15T17:37:54Z

I forgot to mention this failing test, not sure if it’s a random flakiness problem that’s unrelated to this PR though as I don’t see any obvious change here that would relate 🤔

DingoEatingFuzz · 2020-10-15T17:45:29Z

Yeah, this is the first time I've seen that test file. I cleaned up everything other than the SVG bugs, so I'll push that now to see what Circle says this time.

DingoEatingFuzz · 2020-10-15T18:26:20Z

Okay @backspace, ready for re-review.

A few things I won't address right now (but feel free to disagree with me):

M/C alignment in firefox: turns out firefox doesn't support alignment-baseline so I'll need a new approach here. Seems too minor to delay things further.
Mirage scenarios that don't overload nodes causing weird lines drawn to visually the wrong clients: definitely this is unfortunate, but it can't happen on a real cluster, so I think it's low priority.
Mhz vs. MHz: I'll do this in a follow up PR across everything, including instances like primary metric which already had this problem.

backspace

ya I’m definitely fine with you ignoring those things! I had a question about a remaining change to the environment file but it’s looking good 😀

backspace · 2020-10-15T18:25:03Z

ui/app/styles/charts/topo-viz-node.scss

+    transform: translate(50%, 50%);
+    text-anchor: middle;
+    alignment-baseline: central;
+  }


whew, sad times but nice fix

backspace · 2020-10-15T18:25:48Z

ui/app/styles/components/legend.scss

+      margin-left: 1.5em;
+    }
+  }
+}


looks good 🎉

backspace · 2020-10-15T18:27:57Z

ui/config/environment.js

@@ -25,10 +25,10 @@ module.exports = function(environment) {

    APP: {
      blockingQueries: false,
-      mirageScenario: 'topoSmall',
+      mirageScenario: 'smallCluster',


This still has blockingQueries changed to false by default, just making sure that’s by choice 🤓

It wasn't...but I also like it? Ha. Blocking queries + Mirage is kinda janky anyway.

Oh shoot, nvm, I'll flip it back. I forget that this impacts non-mirage too.

backspace · 2020-10-15T18:29:04Z

ui/mirage/scenarios/topo.js

-
-export function topoMediumBatch(server) {}
-
-export function topoMediumVariadic(server) {}


There’s another empty topoSmallProblems higher up… not a big deal but might be worth removing. I’m viewing “changes since” or whatever hence me not commenting on that line 🙃

backspace · 2020-10-15T18:29:39Z

ui/tests/helpers/glimmer-factory.js

+// Used in glimmer component unit tests. Glimmer components should typically
+// be tested with integration tests, but occasionally individual methods or
+// properties have logic that isn't coupled to rendering or the DOM and can
+// be better tested in a unit fashion.


thanks, now I understand why I might use this 😀

github-actions · 2022-12-14T02:18:52Z

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

DingoEatingFuzz force-pushed the f-ui/topo-viz branch from 346cbd4 to 28a7cc2 Compare October 14, 2020 08:05

vercel bot deployed to Preview October 14, 2020 08:05 View deployment

DingoEatingFuzz added this to the 1.0 milestone Oct 14, 2020

vercel bot deployed to Preview October 15, 2020 09:54 View deployment

DingoEatingFuzz added 24 commits October 15, 2020 02:54

Scaffold a new topology page

753bfbf

Small cluster example scenario for the topo viz

1e9e2a1

Scaffold topo viz components

6d9f59f

Add ember modifiers addon

b6867ef

Quick window resize modifier

a003a8a

Prototype of the topo viz

773404a

- Plot all datacenters - For each datacenter, plot all nodes - For each node, plot all allocations by memory and cpu - For empty nodes, highlight the emptiness - When hovering over allocations, give them visual focus

Fix factory bug that made it so pending allocs had no resources

7cf3b74

Start click interaction for topo viz allocs

b3ddd1f

Tweak topo scenario

606910f

Fleshing out the first prototype of the topology visualization

36df145

Touch up topo viz interactions

0b3a78b

Add cluster details to the topology page

c2b7f7c

Super rough allocation details, needs some style love

f5c3d0e

Associate sibling allocations by drawing lines

f56f631

Guard against undefined denominators

290f7d8

Updated alloc and cluster details on topo page

3cd464d

Medium scale topo scenario

98dd258

CSS Grid based masonry layout

2704c3e

TBH, it's buggy and I don't like it.

Add more variety to the node heights

e9626ce

Make the alloc select render path more efficient by not needlessly re…

9991dc6

…computing data

Connect the memory and cpu rectangles

37cc9b7

A better loading screen for the topo viz while nodes load

be2d0fc

Second attempt at a masonry layout

a9e2917

DingoEatingFuzz force-pushed the f-ui/topo-viz branch from 447fdcc to 73dbb3e Compare October 15, 2020 09:54

DingoEatingFuzz requested a review from backspace October 15, 2020 09:54

DingoEatingFuzz marked this pull request as ready for review October 15, 2020 09:55

vercel bot deployed to Preview October 15, 2020 09:58 View deployment

Some light topology acceptance tests

b5809d4

DingoEatingFuzz force-pushed the f-ui/topo-viz branch from 73dbb3e to b5809d4 Compare October 15, 2020 10:06

vercel bot deployed to Preview October 15, 2020 10:07 View deployment

backspace suggested changes Oct 15, 2020

View reviewed changes

Treat legend term pairs as single wrapping elements

229fa29

vercel bot deployed to Preview October 15, 2020 17:45 View deployment

vercel bot deployed to Preview October 15, 2020 18:03 View deployment

vercel bot deployed to Preview October 15, 2020 18:08 View deployment

DingoEatingFuzz requested a review from backspace October 15, 2020 18:23

backspace approved these changes Oct 15, 2020

View reviewed changes

DingoEatingFuzz added 6 commits October 15, 2020 11:45

Reset the standard environment values

e540312

Describe the glimmer-factory better including the motive

223011e

Remove the scenarios I didn't need/didn't get to

2cf4b5d

Factor out the common sum aggregator used in the topology controller

0d920ae

Work around Safari's lack of text transform support

214ce4a

Typo

8f94a98

DingoEatingFuzz force-pushed the f-ui/topo-viz branch from f48bd75 to 8f94a98 Compare October 15, 2020 18:46

vercel bot deployed to Preview October 15, 2020 18:46 View deployment

DingoEatingFuzz merged commit 5302ab0 into master Oct 15, 2020

DingoEatingFuzz deleted the f-ui/topo-viz branch October 15, 2020 18:58

DingoEatingFuzz requested a review from a team October 15, 2020 18:59

github-actions bot locked as resolved and limited conversation to collaborators Dec 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UI: Topology Visualization #9077

UI: Topology Visualization #9077

DingoEatingFuzz commented Oct 13, 2020 •

edited

Loading

github-actions bot commented Oct 13, 2020 •

edited

Loading

github-actions bot commented Oct 13, 2020 •

edited

Loading

backspace left a comment

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

DingoEatingFuzz Oct 15, 2020

backspace Oct 15, 2020

DingoEatingFuzz Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

DingoEatingFuzz Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

DingoEatingFuzz Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace commented Oct 15, 2020

DingoEatingFuzz commented Oct 15, 2020

DingoEatingFuzz commented Oct 15, 2020

backspace left a comment

backspace Oct 15, 2020

backspace Oct 15, 2020

backspace Oct 15, 2020

DingoEatingFuzz Oct 15, 2020

DingoEatingFuzz Oct 15, 2020 •

edited

Loading

backspace Oct 15, 2020

backspace Oct 15, 2020

github-actions bot commented Dec 14, 2022


		export function topoMediumBatch(server) {}

		export function topoMediumVariadic(server) {}

UI: Topology Visualization #9077

UI: Topology Visualization #9077

Conversation

DingoEatingFuzz commented Oct 13, 2020 • edited Loading

TODO

github-actions bot commented Oct 13, 2020 • edited Loading

Ember Asset Size action

github-actions bot commented Oct 13, 2020 • edited Loading

Ember Test Audit comparison

backspace left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

backspace commented Oct 15, 2020

DingoEatingFuzz commented Oct 15, 2020

DingoEatingFuzz commented Oct 15, 2020

backspace left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DingoEatingFuzz Oct 15, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Dec 14, 2022

DingoEatingFuzz commented Oct 13, 2020 •

edited

Loading

github-actions bot commented Oct 13, 2020 •

edited

Loading

github-actions bot commented Oct 13, 2020 •

edited

Loading

DingoEatingFuzz Oct 15, 2020 •

edited

Loading