Integrations common lib #1028

v-zhuravlev · 2023-07-24T13:15:43Z

This commonlib implements
the following panel groups:
generic as base group, then:

system
cpu
memory
disk
network

Inside each group, the following panel types can be found:

timeSeries
table
statusHistory
stat

Also implements common grafana annotations prototypes (such as reboot event)
Adds some util functions to manipulate labels and variables

All panels have new() method to create this type of panel and stylize() method now that can be used to 'mutate' old panels or change style of existing panels.
You can also use it as stylize(allLayers=false) to ignore base styles and apply only top layer. This can be handy when you want to cherry-pick different style sets from different panels to compose new combination.

For actual use of this lib see #1050.

malcolmholmes · 2023-07-24T16:24:40Z

common-lib/common/g.libsonnet

@@ -0,0 +1 @@
+import 'github.com/grafana/grafonnet/gen/grafonnet-latest/main.libsonnet'


I would suggest pinning to a specific Grafana version and let Grafana handle upgrading.

hold to v10.0.0

malcolmholmes · 2023-07-24T16:27:53Z

common-lib/common/panels.libsonnet

+local g = import 'g.libsonnet';
+
+{
+  stat: import 'panels/stat/main.libsonnet',


does this import work? Won't it be looking in the JSONNET_PATH for this lib? I'd say ./panels/... would be better.

updated, thnx

rgeyer

Several questions, mostly open-ended for the sake of discussion.

I generally like the idea that we're being opinionated, as that should be the purpose of this library. This will require fairly comprehensive documentation to explain the actual result of these assumptions/opinions. For example, in the stats info panel, I agree with the assertions, but the description of "simple info panel with text or count" falls short in fully describing what type of panel you get, and when you'd use it.

An illustrated guide with screenshots and use-cases would likely be appropriate.

common-lib/common/panels/all/base.libsonnet

common-lib/common/panels/stat/base.libsonnet

common-lib/common/panels/stat/percentage.libsonnet

common-lib/common/panels/timeSeries/network/base.libsonnet

v-zhuravlev · 2023-08-04T16:45:32Z

Several questions, mostly open-ended for the sake of discussion.

I generally like the idea that we're being opinionated, as that should be the purpose of this library. This will require fairly comprehensive documentation to explain the actual result of these assumptions/opinions. For example, in the stats info panel, I agree with the assertions, but the description of "simple info panel with text or count" falls short in fully describing what type of panel you get, and when you'd use it.

An illustrated guide with screenshots and use-cases would likely be appropriate.

Agree, more detailed descriptions are necessary.

common-lib/common/panels.libsonnet

common-lib/common/panels/all/stat/info.libsonnet

rgeyer

Looking very good.. I have a few nits/questions. Enough that I'll refrain from approving just yet.

The biggest concern is that there seem to be several commits/changed files which don't relate to this PR.

Tremendous work! Thank you for the dedication to getting this finished!

rgeyer · 2023-10-16T18:42:33Z

common-lib/CONTRIB.md

+All panels in this lib should implement one of the following methods: 
+
+- `panel.new(title,targets,description)` - creates new panel. List of arguments could vary;
+- `panel.stylize(allLayers=true)` - directly applies this panel style to existing panel. By default includes all layers of styles. To apply only top layer, set allLayers=false. This mode is useful to cherry-pick style layers to create new style combination.


Can we document the concept of "layers" here a bit more thoroughly? The concept is not immediately apparent from the CONTRIB.md or README.md.

It looks like what is meant is if you, for example, apply stylize from panels/network/timeseries/packets/, with allLayers you will effectively apply not only the styles specific to a network packets timeseries (ingress above 0, egress below 0), but also the default style from panels/generic/timeseries?

Possibly an architecture diagram of the layout of this lib would be useful.

Also, I would advise that this get added to a follow-up PR, and we create an issue to track. No need to hold the merging of this PR.

It looks like what is meant is if you, for example, apply stylize from panels/network/timeseries/packets/, with allLayers you will effectively apply not only the styles specific to a network packets timeseries (ingress above 0, egress below 0), but also the default style from panels/generic/timeseries?

correct. I'll try to expand documentation once we came to terms.

aerospike-mixin/.lint

common-lib/common/panels/all/base.libsonnet

rgeyer · 2023-10-16T18:52:08Z

common-lib/common/annotations/fatal.libsonnet

+    title,
+    target,
+  ):
+    super.new(title, target)


There's no built-in logic for identifying fatal/critical events, which I believe is intentional. A user would define this by providing a target with a query which only surfaces fatal and critical events, yes?

yes, you got it right. We just provide this annotation.fatal as prototype, where color choice is enforced for now.

rgeyer · 2023-10-16T18:54:36Z

common-lib/common/annotations/fatal.libsonnet

+    target,
+  ):
+    super.new(title, target)
+    + annotation.withIconColor('light-purple')


I'm a fan of purple... But.. Wouldn't fatal or critical events be red/orange?

I haven't gotten through the other panels yet, but perhaps this is part of the embedded design language that's used throughout the library?

If so, this is another good opportunity for expanded documentation. I.E. A table of colors, and where/why they're used. Also examples on how to override those values if you wish.

Again, this is probably appropriate for a followup PR, with an issue to track.

Just following colors used in grafana itself for autocoloring loki logs: https://github.com/grafana/grafana/blob/fe9f52097bb4717a7ebfac17a62f399a2ce0ba1b/docs/sources/explore/index.md#log-level

Ahh perfect! Still worth documenting it (again, in a future PR).

I have a bias with purple. It's my favorite color, so I don't immediately associate it with "bad things". But I'm always reminded that it's "darker" than red, and thus more severe. :)

common-lib/common/annotations/reboot.libsonnet

common-lib/common/panels/disk/table/usage.libsonnet

common-lib/common/panels/generic/base.libsonnet

common-lib/common/panels/generic/timeSeries/percentage.libsonnet

common-lib/common/panels/generic/timeSeries/topk_percentage.libsonnet

rgeyer

Thank you for the regular updates and getting this sorted out. I think we're ready to merge this!

Again tremendous work! Great job!

Signed-off-by: Vitaly Zhuravlev <v-zhuravlev@users.noreply.github.com>

Co-authored-by: Ryan Geyer <me@ryangeyer.com>

Fix available and free calculation

Co-authored-by: Ryan Geyer <me@ryangeyer.com>

v-zhuravlev mentioned this pull request Jul 24, 2023

Add node-observ-lib grafana/node_exporter#25

Merged

malcolmholmes reviewed Jul 24, 2023

View reviewed changes

v-zhuravlev requested review from malcolmholmes, rgeyer and Duologic July 26, 2023 15:08

rgeyer reviewed Aug 2, 2023

View reviewed changes

rgeyer reviewed Aug 15, 2023

View reviewed changes

common-lib/common/panels.libsonnet Outdated Show resolved Hide resolved

common-lib/common/panels/all/stat/info.libsonnet Outdated Show resolved Hide resolved

v-zhuravlev force-pushed the vzhuravlev/common-lib branch 2 times, most recently from 6c99609 to a15b493 Compare September 27, 2023 17:57

v-zhuravlev marked this pull request as ready for review October 11, 2023 17:29

v-zhuravlev requested a review from rgeyer October 11, 2023 17:56

rgeyer reviewed Oct 16, 2023

View reviewed changes

v-zhuravlev force-pushed the vzhuravlev/common-lib branch from d31a4bc to 916695f Compare October 17, 2023 14:31

v-zhuravlev requested a review from rgeyer October 17, 2023 14:43

rgeyer approved these changes Oct 17, 2023

View reviewed changes

v-zhuravlev and others added 14 commits October 18, 2023 16:22

Init lib

3430ad1

add grafonnet import

5b574af

Add panels in common lib

aac4467

Update Makefile

6e6a921

Add entrypoint for panels

baf69ab

Fix to v10.0.0

e42c21b

Switch from function() to new()

d7c45aa

jsonnetfmt

148a906

Update stat panels

5e93a5f

Drop fixed interval for all intervals

947e73d

Signed-off-by: Vitaly Zhuravlev <v-zhuravlev@users.noreply.github.com>

Shuffle lib structure and add comments

a775ca1

Update common-lib/common/panels.libsonnet

2f4d4df

Co-authored-by: Ryan Geyer <me@ryangeyer.com>

Add cpu usage stat

853b407

Add stylize function

7fcb5f1

v-zhuravlev and others added 25 commits October 18, 2023 16:22

Add disk panels

0e23766

Add default descriptions

382d45d

Merge disk and filesystem

974ff27

Remove bigger text in stat panels

fe90390

Fix disks styles

5a6d0b2

Remove unused

dd0e353

Update memory panels

fb90335

Fix disk usage table

d1f4cc7

Fix available and free calculation

Add utils (from windows lib)

f695413

Add link to utils

994137f

Update panels

f6758a1

Add contrib

6d3a42d

Rename 'all' to 'generic'

7247c57

Update panels

2c9bbf5

Update ts stylize

bc0fa62

Update stat panels stylize

ef263c4

Fix ts stylize

83b36a8

Add table

cb1fb97

Update status-history

dede26b

Update README

869f93d

lint-fmt

b9aa2f1

Drop title from base class args

6d776a3

Update common-lib/common/panels/generic/base.libsonnet

e1aa770

Co-authored-by: Ryan Geyer <me@ryangeyer.com>

Update common-lib/common/panels/generic/timeSeries/percentage.libsonnet

cb079c6

Co-authored-by: Ryan Geyer <me@ryangeyer.com>

Remove comments

cfc250f

v-zhuravlev force-pushed the vzhuravlev/common-lib branch from 0edca29 to cfc250f Compare October 18, 2023 08:22

v-zhuravlev enabled auto-merge (squash) October 18, 2023 08:23

v-zhuravlev merged commit cfd1dfe into master Oct 18, 2023

v-zhuravlev deleted the vzhuravlev/common-lib branch October 18, 2023 08:23

v-zhuravlev mentioned this pull request Nov 29, 2023

Refactor node-mixin as node-observ-lib prometheus/node_exporter#2861

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrations common lib #1028

Integrations common lib #1028

v-zhuravlev commented Jul 24, 2023 •

edited

Loading

malcolmholmes Jul 24, 2023

v-zhuravlev Jul 26, 2023

malcolmholmes Jul 24, 2023

v-zhuravlev Jul 26, 2023

rgeyer left a comment

v-zhuravlev commented Aug 4, 2023

rgeyer left a comment

rgeyer Oct 16, 2023

v-zhuravlev Oct 17, 2023

rgeyer Oct 16, 2023

v-zhuravlev Oct 17, 2023

rgeyer Oct 16, 2023

v-zhuravlev Oct 17, 2023

rgeyer Oct 17, 2023

rgeyer left a comment

		@@ -0,0 +1 @@
		import 'github.com/grafana/grafonnet/gen/grafonnet-latest/main.libsonnet'

Integrations common lib #1028

Integrations common lib #1028

Conversation

v-zhuravlev commented Jul 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rgeyer left a comment

Choose a reason for hiding this comment

v-zhuravlev commented Aug 4, 2023

rgeyer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rgeyer left a comment

Choose a reason for hiding this comment

v-zhuravlev commented Jul 24, 2023 •

edited

Loading