Update landing page / dashboard

[kinow]

Part of #94, opening the way for #139. This pull request removes all the links from the sidebar, but the Suites/Graph view links.

The idea is that this way, once GScan works begin, the other links such as Hub, User, etc have already been moved elsewhere (the login page was done in #147).

Tried to follow the example from the current design document (#87 ), but my SvgFu fails me @oliver-sanders . Do you reckon it would be easy to create the arrows and connect the nodes, as in the sketch?

I used the components Task and Job and the result is close to the sketch. There is some example code in this PR with the ID of the Task element, which contains the SVG node.

Then, there is a JS function pending implementation where I was going to start working on creating the lines/arrows.

The issue that I have now, is that I haven't done that without a canvas? I found a few people with ideas like creating an SVG element on the fly and just positioning it on the UI, or creating a DIV and hacking its X/Y/Z, etc.

But I thought at this stage it would be better to show you what I've done today and see if it's going in a good direction, or if you had something else in mind.

Note: not a complete solution for the #94 , as that issue has discussion about allowing users to customize their dashboards. However, this could very well be the default dashboard, when the user has not customized the UI.

[kinow]

What it looks like on Firefox now:

Medium (forgot to center align the Cylc Hub link):

Small (annoying issue with alignment when it's too small, but would prefer to investigate it later)

[kinow]

@oliver-sanders also note:

• I've replaced Suite Design Guide by the link to the CUG, as our docs currently have only the latter, and say that the former has been incorporated into it.
• I've also removed the Stop Server option... alas in JupyterHub there is no link for shutting down. Instead, they capture the click on the button to call a JS function, that then calls the API /shutdown, and on success update the buttons (displaying now Start My Server). Not quite easy to reproduce that feature on Cylc UI without some extra work to get some JS code calling the API as they are doing... but for now I thought it would be easier to focus just on the dashboard here.

[codecov-io]

Codecov Report

Merging #149 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #149   +/-   ##
=======================================
  Coverage   36.47%   36.47%           
=======================================
  Files          13       13           
  Lines         170      170           
=======================================
  Hits           62       62           
  Misses        108      108

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d5e1b1...7b56f29. Read the comment docs.

[kinow]

(oh, also the progress on the Task didn't work, so I've added a post-it to investigate as it appears to be happening on master as well).

[hjoliver]

@kinow and @oliver-sanders - I just noticed the "Dashboard" addition to the design doc (not sure how I missed it earlier, sorry). I'm not convinced the fake dependency graph is a good thing - the "dependencies" are a bit tenuous at best, aren't they? It might just be confusing. In which case, I actually like @kinow's arrowless version above better - the task and job icons are more obviously just there for aesthetic reasons (although even then, it might be better to use standard icons such as a book for the documentation link, and a "hub" for the Hub, etc.?

[oliver-sanders]

my SvgFu fails me @oliver-sanders . Do you reckon it would be easy to create the arrows and connect the nodes, as in the sketch?

Kinda difficult, we should use the Graph view code to do this?

I've replaced Suite Design Guide by the link to the CUG, as our docs currently have only the latter, and say that the former has been incorporated into it.

Can just link to the "Suite Design Guide" within the CUG.

I've also removed the Stop Server option... alas in JupyterHub there is no link for shutting down.

Sure, we can attack this one another day.

I'm not convinced the fake dependency graph is a good thing

Fair enough! It's a bit funky, the dashboard I put up there was really just illustrative! I think it would be good to have all of those bits in there somehow but design might need some work!

[sadielbartholomew]

Great work, @kinow! This is a great start for the dashboard.

From a skim read over the code, it seems the workflow stats displayed are just placeholders ( workflows: [{ text: 'Running', count: 0 } ... etc.)? I'm not sure how long you expect it will be before we can get those real-time figures through (it seems simple enough as the data required is already there e.g. for GScan, & just needs to be coded in to be computed & displayed there?) but if it might be a while, perhaps we could add a note that they are just dummy figures? Just in case someone tries out the UI & thinks that the dashboard stats are not picking up on their suites.

Just to add my thoughts RE the design:

I'm not convinced the fake dependency graph is a good thing ...

... the task and job icons are more obviously just there for aesthetic reasons (although even then, it might be better to use standard icons such as a book for the documentation link, and a "hub" for the Hub, etc.?

I do agree with @hjoliver in that I am not keen on the link styling à la Graph View (not just the dependencies but also the nodes themselves with the task &/or job icons). It's an interesting idea, & I do appreciate more creative ideas, but I think there is a lot of potential for it to be confusing. Also I don't think it works well with so few links; if there were, say, >10 links, it could maybe work. And I think it best to reserve the icons solely for real tasks and jobs, so users don't mistakenly interpret those icons for real tasks/jobs when trying to interpret the state of suite(s) they are monitoring at a glance, perhaps over several browser tabs or windows.

Besides the design, I don't think we should give much prominence to the links. In particular, emphasising the link to the documentation could give off the impression that users should be expected to need to cross-reference with it often. In fact, if we highlight the docs prominently, I think we would have to indicate in some way (a simple text note?) that the documentation is not required for, & does not cover, navigating the UI (which obviously should not need instructions, as that would be a sign we have a bad UI), in case they think that (& it is not a wild step to think so, if the docs are linked from the dashboard!).

I think the links should be clear & accessible, but not one of the main features highlighted on the dashboard page, even for this early-stage version of it. For now I'd prefer, if anything, to add more on the basic workflow statistics side (there's quite a lot we could display, even at this stage) or something different entirely.

So my suggestion for now is to just use a simple list (with no icons) for the links.

[kinow]

Hi @sadielbartholomew

So my suggestion for now is to just use a simple list (with no icons) for the links.

+1, and I think @hjoliver agrees too to drop the design from the sketch. I liked it, but doesn't seem very easy to implement, but we can revisit it later.

Will update it once I've found the Vuetify component to use for this list.

[kinow]

Simpler v-list's now:

Using two columns of 6 each (a row has 12 columns) when on large screens. For medium/small screens, we're using columns of 12.

And @oliver-sanders

Can just link to the "Suite Design Guide" within the CUG.

Done!

[kinow]

@sadielbartholomew

From a skim read over the code, it seems the workflow stats displayed are just placeholders ( workflows: [{ text: 'Running', count: 0 } ... etc.)? I'm not sure how long you expect it will be before we can get those real-time figures through (it seems simple enough as the data required is already there e.g. for GScan, & just needs to be coded in to be computed & displayed there?) but if it might be a while, perhaps we could add a note that they are just dummy figures? Just in case someone tries out the UI & thinks that the dashboard stats are not picking up on their suites.

I thought I wouldn't have this information already digested. I thought I would have find it somewhere in Vuex, then add some extra code for doing that.

But let me investigate and see if I can manage to retrieve this data from the WorkflowService (as there's already a pending pull request to eliminate the SuiteService 🎉 ).

[kinow]

I don't think we have that information ready when the dashboard is displayed. Here's the Vuex store when the application is loaded (with the Hub + proxy) and I have already a workflow running (cylc run --no-detach --verbose --hold five):

Then if I open the /#/workflows route, I can see that the workflows object has been loaded:

And even though it's Held, I don't see an easy way to retrieve this information from the workflow. Only when I open /#/workflows/five, then the status is correctly displayed:

So we still need more work before we can plug that data into the dashboard IMO. What do you think @sadielbartholomew ?

[kinow]

Rebased, waiting to confirm it's good to be merged without the integration with the Vuex store values, but other than that, should be ready for review.

[hjoliver]

@kinow - current (cylc-7) scan (GUI or CLI) shows what information should be available without opening a specific workflow, e.g.:

[oliverh@niwa-1007823l cylc-flow]$ cylc scan -f
test1 oliverh@localhost:43054
   Title:
      "gcylc task state color theme demo"
   URL:
      (no URL)
   Group:
      (no group)
   Description:
      "Generate a lot of possible task states,
       to show what they look like live in gcylc."
   Task state totals:
      queued:4 runahead:1 running:2 submit-retrying:4 submitted:3 succeeded:4 waiting:4
      2012-08-08T00Z queued:1 running:2 submit-retrying:2 submitted:1 succeeded:3 waiting:1
      2012-08-09T00Z queued:3 submit-retrying:2 submitted:1 succeeded:1
      2012-08-10T00Z submitted:1 waiting:3
      2012-08-11T00Z runahead:1

test2 oliverh@localhost:43067
   Title:
      "gcylc task state color theme demo"
   URL:
      (no URL)
   Group:
      (no group)
   Description:
      "Generate a lot of possible task states,
       to show what they look like live in gcylc."
   Task state totals:
      failed:1 runahead:8 running:1 submit-failed:1 succeeded:6 waiting:1
      2012-08-11T00Z succeeded:1
      2012-08-12T00Z failed:1 running:1 submit-failed:1 succeeded:5 waiting:1
      2012-08-13T00Z runahead:8

i.e. total number of tasks in each state, for each workflow.

Not sure if we have the same in Cylc 8 yet though??

[hjoliver]

Of course that's not actually very helpful to the proposed dashboard content. I don't think suite scan returns whether a whole suite is held at the moment (although that would be a simple extension).

[hjoliver]

You should be able to get the number of running suites, just by counting the number of suites returned by the scan.

[kinow]

@kinow - current (cylc-7) scan (GUI or CLI) shows what information should be available without opening a specific workflow, e.g.:
i.e. total number of tasks in each state, for each workflow.
Not sure if we have the same in Cylc 8 yet though??

Not sure as well. I thought these totals would represent the Workflows totals (as they are under Workflows).

So instead of displaying how many tasks you have in each state, it would show how many workflows you have in each state?

Of course that's not actually very helpful to the proposed dashboard content. I don't think suite scan returns whether a whole suite is held at the moment (although that would be a simple extension).

I think when you go to a /#/workflows/$workflow-name/, the GraphQL query returned to that View contains the workflow state.

You should be able to get the number of running suites, just by counting the number of suites returned by the scan.

Yup. The current issue is more of organization. I believe we are missing the GScan component replacing the sidebar (Drawer). When that happens, we will have a single query being executed to bring all the workflows, then further actions in the UI will augment that query (if I understand @oliver-sanders 's current GraphQL Proxy work).

We don't have that as we started with a menu, showing a table with workflows, then another view with a single workflow... so our data is being populated only when the view needs, and that's why we have the empty Vuex stores when we open the Dashboard.

[oliver-sanders]

We can get the information for this view via the WorkflowService, just register a query like this:

workflows {
    name,
    status
}

This will overlap with the GScan query so the WorkflowService will merge the queries and manage the subscription.

[kinow]

@oliver-sanders when we move GScan.vue to the sidebar as in the sketches, won't that fix this issue as well? We will have to add a new polling to the Dashboard.vue, and later remove the polling and the query, once we move the GScan.vue.

I started removing the menu items and clearing up the left-hand area so that GScan.vue could be done soon - planning to start working on that after our meeting on Tuesday, just need to check who will work on what, etc 😬

[oliver-sanders]

when we move GScan.vue to the sidebar as in the sketches, won't that fix this issue as well

For view isolation it makes sense for each view to request all of the data it needs rather than relying on it being provided by another view. For example:

• What if we wanted some information in the dashboard not present in GScan?
• What if the GScan sidebar is hidden and we do some neat jiggery-pokery to suspend it's subscriptions?

We will have to add a new polling to the Dashboard.vue

The WorlflowService does the polling for you?

[kinow]

For me the Dashboard would just request data from the Vuex central store.... as when Cylc UI starts it will have to do that large query anyway I think.

If we decide to add a sort of tray icon, it would mean that the TrayIcon would be asking the service to retrieve the data too...

But I thought we would have a single websocket (now polling) putting all the data into the Vuex store... views and components would ask the service to augment the query and bring more data... but would query the store and filter locally uf necessary.

Will have another go tomorrow when I'm in front of the PC. Maybe I'm misunderstanding how it would work 👍

[oliver-sanders]

I think I'm misunderstanding what you're writing but I'll have a go at answering...

For me the Dashboard would just request data from the Vuex central store

But how does the data get into the Vuex central data store, what is requesting it?

when Cylc UI starts it will have to do that large query anyway I think.

???

When the UI starts the different views register subscriptions and the WorkflowService starts to gather the required data? Is there any need for a special query which can't be handled by subscriptions?

If we decide to add a sort of tray icon, it would mean that the TrayIcon would be asking the service to retrieve the data too...

???

Not sure what you mean.

|

The Dashboard is a special "view" in the same way that GScan is a "view" so I would have thought it would request its data in the same way?

At present the WorkflowService only has a "subscribe" interface but should also have a "one-off-request" or "polling" interface for data which we don't want a regular subscription for.

[kinow]

Sorry, wrote from mobile without being very clear. The example of the TrayIcon would be if we wanted to display the same information as the Dashboard, but instead in a sort of StatusBar or TrayIcon component, also in the same page/view/URL (i.e. from /).

(just to make sure it's clear: the Dashboard.vue and GScan.vue are both components. At the moment there is a route for both, so they are components bound to views in Vuejs. The folder views that we have in our project is created by Vuetifyjs (vue-cli gives you public and src, the latter having only src/assets/ and /src/components, nothing more).)

I thought we would use GScan.vue in the Drawer.vue component, to render the output of gscan in the left sidebar, as in the sketches. So the Route / will be bound to Dashboard.vue... but that means that we will have the Toolbar, Dashboard, Footer, Drawer, GScan, and other components loaded. The view that is handling / will have to

1. have already loaded the User information in the Router for authentication (this will go to the router later, but it's in UserProfile right now, mea culpa)
2. load data for GScan and Dashboard

For the item 2, I was expecting that the same query that will bring all my workflows could be re-used for the Dashboard. We would use one of VueRouter lifecycle callbacks to fetch the data before the view is loaded... the both GScan and Dashboard could simply consume the data from Vuex store.

My rationale for that, was that it kind of made sense to request it once instead of multiple times from a service. And if we go and add GScan + N tabs opened (the JupyterLab approach), each with one component, would each component then send a request to the service? And would each request become a request-response in WebSocket?

Time for last pill of the day to cure from the cold, and start to get groggy again, so will reply tomorrow and avoid blabbing more nonsense here (I could be completely wrong here, just trying to explain from where I was coming). Hope it's more clear now, and sorry for the previous unclear comment.

B

[kinow]

(tomorrow I will probably just use the service here so that we can solve this issue and move this discussion elsewhere as I think it's getting out of control and mainly due to me not understanding how we should put pieces together 👍 )

[sadielbartholomew]

I will probably just use the service here so that we can solve this issue and move this discussion elsewhere

Oops, sorry @kinow I didn't mean to start an in-depth discussion on data provision with my comment about the workflow statistic placeholders! Now I realise was underestimating the ease of getting that data, so no worries at all, & I think a new issue is definitely a more appropriate place for discussions on that.

Great work on the dashboard, the UI is looking more & more like the mock-ups by the week 🎆

[kinow]

Data added to Dashboard with WorkflowService.

What it looks like now:

The Dashboard.vue uses stats local dict/object, with keys for the Text and Count to be displayed (can be improved later). These values are set by a watch, that is called whenever the workflows element is touched.

Will create follow up tickets 👍

[oliver-sanders]

Looks good.

Can go in before or after cylc/cylc-flow#3267

[hjoliver]

(This can go in as far as I'm concerned, @kinow ... some conflicts though).

[kinow]

Conflicts resolved, thanks @hjoliver !