Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrations common lib #1028

Merged
merged 69 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
3430ad1
Init lib
v-zhuravlev Jul 21, 2023
5b574af
add grafonnet import
v-zhuravlev Jul 21, 2023
aac4467
Add panels in common lib
v-zhuravlev Jul 24, 2023
6e6a921
Update Makefile
v-zhuravlev Jul 24, 2023
baf69ab
Add entrypoint for panels
v-zhuravlev Jul 24, 2023
e42c21b
Fix to v10.0.0
v-zhuravlev Jul 26, 2023
d7c45aa
Switch from function() to new()
v-zhuravlev Jul 26, 2023
148a906
jsonnetfmt
v-zhuravlev Jul 26, 2023
5e93a5f
Update stat panels
v-zhuravlev Jul 26, 2023
947e73d
Drop fixed interval for all intervals
v-zhuravlev Aug 15, 2023
a775ca1
Shuffle lib structure and add comments
v-zhuravlev Aug 15, 2023
2f4d4df
Update common-lib/common/panels.libsonnet
v-zhuravlev Aug 15, 2023
853b407
Add cpu usage stat
v-zhuravlev Aug 15, 2023
7fcb5f1
Add stylize function
v-zhuravlev Aug 15, 2023
ec9c420
Add cpu percentage
v-zhuravlev Aug 15, 2023
3e4b5d0
Fix tooltip and legend
v-zhuravlev Aug 15, 2023
a2edc2e
Network base: withAxisCenteredZero when withNegateOutPackets
v-zhuravlev Aug 15, 2023
7e9f32f
Fix timeSeries style
v-zhuravlev Aug 15, 2023
5d48a2e
Fix reduceOptions:
v-zhuravlev Aug 28, 2023
b533802
DROP grafonnet inheritance for basic panels
v-zhuravlev Aug 29, 2023
7966cdf
Add filesystem usage table
v-zhuravlev Aug 29, 2023
b1d644a
Add common annotation
v-zhuravlev Aug 29, 2023
781c316
Add filesystem table with input validtion
v-zhuravlev Sep 5, 2023
28b1fa9
Add threshold
v-zhuravlev Sep 6, 2023
e6a73d5
Add gradient 'scheme' to percentage
v-zhuravlev Sep 6, 2023
1a95561
Add cpu.stat
v-zhuravlev Sep 6, 2023
07c9360
update cpu.timeSeries
v-zhuravlev Sep 6, 2023
cc295d5
Add filesystems.stat
v-zhuravlev Sep 6, 2023
d4083ad
Upate network.timeSeries
v-zhuravlev Sep 6, 2023
53493ed
Update disk.timeSeries
v-zhuravlev Sep 6, 2023
90739d9
Update panels
v-zhuravlev Sep 6, 2023
bf5ac34
Add topkPercentage
v-zhuravlev Sep 7, 2023
12e239e
Add table panels
v-zhuravlev Sep 8, 2023
73d7cc0
Update to work with latest mixin
v-zhuravlev Sep 8, 2023
e0d04c3
Remove duplicated
v-zhuravlev Sep 20, 2023
c54fa7b
add panel wide datasource
v-zhuravlev Sep 20, 2023
52a4583
add serviceFailed annotation
v-zhuravlev Sep 20, 2023
d8203a1
Add new annotations
v-zhuravlev Sep 21, 2023
7570070
Add datalink to topk_percentage
v-zhuravlev Sep 21, 2023
88a38dd
Update topk
v-zhuravlev Sep 21, 2023
883e6f5
Fix 'all' panel
v-zhuravlev Sep 21, 2023
e18b1f6
Add util by mode
v-zhuravlev Sep 21, 2023
643747a
switch to bytes from decbytes
v-zhuravlev Sep 21, 2023
ad669a8
Add statusHistory panel
v-zhuravlev Sep 22, 2023
0e23766
Add disk panels
v-zhuravlev Sep 22, 2023
382d45d
Add default descriptions
v-zhuravlev Sep 27, 2023
974ff27
Merge disk and filesystem
v-zhuravlev Sep 27, 2023
fe90390
Remove bigger text in stat panels
v-zhuravlev Sep 27, 2023
5a6d0b2
Fix disks styles
v-zhuravlev Sep 27, 2023
dd0e353
Remove unused
v-zhuravlev Sep 27, 2023
fb90335
Update memory panels
v-zhuravlev Sep 27, 2023
d1f4cc7
Fix disk usage table
v-zhuravlev Sep 28, 2023
f695413
Add utils (from windows lib)
v-zhuravlev Oct 6, 2023
994137f
Add link to utils
v-zhuravlev Oct 6, 2023
f6758a1
Update panels
v-zhuravlev Oct 10, 2023
6d3a42d
Add contrib
v-zhuravlev Oct 10, 2023
7247c57
Rename 'all' to 'generic'
v-zhuravlev Oct 10, 2023
2c9bbf5
Update panels
v-zhuravlev Oct 11, 2023
bc0fa62
Update ts stylize
v-zhuravlev Oct 11, 2023
ef263c4
Update stat panels stylize
v-zhuravlev Oct 11, 2023
83b36a8
Fix ts stylize
v-zhuravlev Oct 11, 2023
cb1fb97
Add table
v-zhuravlev Oct 11, 2023
dede26b
Update status-history
v-zhuravlev Oct 11, 2023
869f93d
Update README
v-zhuravlev Oct 11, 2023
b9aa2f1
lint-fmt
v-zhuravlev Oct 11, 2023
6d776a3
Drop title from base class args
v-zhuravlev Oct 11, 2023
e1aa770
Update common-lib/common/panels/generic/base.libsonnet
v-zhuravlev Oct 17, 2023
cb079c6
Update common-lib/common/panels/generic/timeSeries/percentage.libsonnet
v-zhuravlev Oct 17, 2023
cfc250f
Remove comments
v-zhuravlev Oct 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions common-lib/CONTRIB.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@

## Panels overview

All panels in this lib should implement one of the following methods:

- `panel.new(title,targets,description)` - creates new panel. List of arguments could vary;
- `panel.stylize(allLayers=true)` - directly applies this panel style to existing panel. By default includes all layers of styles. To apply only top layer, set allLayers=false. This mode is useful to cherry-pick style layers to create new style combination.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we document the concept of "layers" here a bit more thoroughly? The concept is not immediately apparent from the CONTRIB.md or README.md.

It looks like what is meant is if you, for example, apply stylize from panels/network/timeseries/packets/, with allLayers you will effectively apply not only the styles specific to a network packets timeseries (ingress above 0, egress below 0), but also the default style from panels/generic/timeseries?

Possibly an architecture diagram of the layout of this lib would be useful.

Also, I would advise that this get added to a follow-up PR, and we create an issue to track. No need to hold the merging of this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like what is meant is if you, for example, apply stylize from panels/network/timeseries/packets/, with allLayers you will effectively apply not only the styles specific to a network packets timeseries (ingress above 0, egress below 0), but also the default style from panels/generic/timeseries?

correct. I'll try to expand documentation once we came to terms.


Some other methods could be found such as:
- `panel.stylizeByRegexp(regexp)` - attaches style as panel overrides (by regexp);
- `panel.stylizeByName(name)` - attaches style as panel overrides (by name).

## Panels common groups

This library consists of multiple common groups of panels for widely used resources such as CPU, memory, disks and so on.

All of those groups inherit `generic` group as their base.

All panels inherit `generic/base.libsonnet` via `generic/<paneltype>/base.libsonnet`.

23 changes: 23 additions & 0 deletions common-lib/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
JSONNET_FMT := jsonnetfmt -n 2 --max-blank-lines 1 --string-style s --comment-style s

.PHONY: all
all: build

vendor: jsonnetfile.json
jb install

.PHONY: build
build: vendor

.PHONY: fmt
fmt:
find . -name 'vendor' -prune -o -name '*.libsonnet' -print -o -name '*.jsonnet' -print | \
xargs -n 1 -- $(JSONNET_FMT) -i

.PHONY: lint
lint: build
find . -name 'vendor' -prune -o -name '*.libsonnet' -print -o -name '*.jsonnet' -print | \
while read f; do \
$(JSONNET_FMT) "$$f" | diff -u "$$f" -; \
done

33 changes: 33 additions & 0 deletions common-lib/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Grafana integrations common lib

This common library can be used to quickly create dashboards' `panels` and `annotations`.

By using this common library we can 'enforce' common style choices across multiple dashboards and mixins.

## Import

```sh
jb init
jb install https://github.com/grafana/jsonnet-libs/common-lib
```

## Use

### Create new panel

```jsonnet

local commonlib = import 'github.com/grafana/jsonnet-libs/common-lib/common/main.libsonnet';
local cpuUsage = commonlib.panels.cpu.timeSeries.utilization.new(targets=[targets.cpuUsage]);

```

### Mutate exisiting panel with style options

```jsonnet

local commonlib = import 'github.com/grafana/jsonnet-libs/common-lib/common/main.libsonnet';
local cpuPanel = oldPanel + commonlib.panels.cpu.timeSeries.utilization.stylize();
```

See [windows-observ-lib](./windows-observ-lib) for full example.
32 changes: 32 additions & 0 deletions common-lib/common/annotations/base.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
local g = import '../g.libsonnet';
local annotation = g.dashboard.annotation;
{
new(
title,
target,
):
annotation.withEnable(true)
+ annotation.withName(title)
+ annotation.withDatasourceMixin(target.datasource)
{
titleFormat: title,
expr: target.expr,

}
+ (if std.objectHas(target, 'interval') then { step: target.interval } else {}),

withTagKeys(value):
{
tagKeys: value,
},
withValueForTime(value=false):
{
useValueForTime: value,
},
withTextFormat(value=''):
{
textFormat: value,
},


}
14 changes: 14 additions & 0 deletions common-lib/common/annotations/fatal.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
local g = import '../g.libsonnet';
local annotation = g.dashboard.annotation;
local base = import './base.libsonnet';

// Show fatal or critical events as annotations
base {
new(
title,
target,
):
super.new(title, target)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no built-in logic for identifying fatal/critical events, which I believe is intentional. A user would define this by providing a target with a query which only surfaces fatal and critical events, yes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, you got it right. We just provide this annotation.fatal as prototype, where color choice is enforced for now.

+ annotation.withIconColor('light-purple')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a fan of purple... But.. Wouldn't fatal or critical events be red/orange?

I haven't gotten through the other panels yet, but perhaps this is part of the embedded design language that's used throughout the library?

If so, this is another good opportunity for expanded documentation. I.E. A table of colors, and where/why they're used. Also examples on how to override those values if you wish.

Again, this is probably appropriate for a followup PR, with an issue to track.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh perfect! Still worth documenting it (again, in a future PR).

I have a bias with purple. It's my favorite color, so I don't immediately associate it with "bad things". But I'm always reminded that it's "darker" than red, and thus more severe. :)

+ annotation.withHide(true),
}
6 changes: 6 additions & 0 deletions common-lib/common/annotations/main.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
base: import './base.libsonnet',
reboot: import './reboot.libsonnet',
serviceFailed: import './service_failed.libsonnet',
fatal: import './fatal.libsonnet',
}
16 changes: 16 additions & 0 deletions common-lib/common/annotations/reboot.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
local g = import '../g.libsonnet';
local annotation = g.dashboard.annotation;
local base = import './base.libsonnet';

base {
new(
title,
target,
instanceLabels,
):
super.new(title, target)
rgeyer marked this conversation as resolved.
Show resolved Hide resolved
+ annotation.withIconColor('light-yellow')
+ annotation.withHide(true)
+ { useValueForTime: 'on' }
+ base.withTagKeys(instanceLabels),
}
13 changes: 13 additions & 0 deletions common-lib/common/annotations/service_failed.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
local g = import '../g.libsonnet';
local annotation = g.dashboard.annotation;
local base = import './base.libsonnet';

base {
new(
title,
target,
):
super.new(title, target)
+ annotation.withIconColor('light-orange')
+ annotation.withHide(true),
}
1 change: 1 addition & 0 deletions common-lib/common/g.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import 'github.com/grafana/grafonnet/gen/grafonnet-v10.0.0/main.libsonnet'
5 changes: 5 additions & 0 deletions common-lib/common/main.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
annotations: import './annotations/main.libsonnet',
panels: import './panels.libsonnet',
utils: import './utils.libsonnet',
}
31 changes: 31 additions & 0 deletions common-lib/common/panels.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
local g = import './g.libsonnet';

{
generic: {
stat: import './panels/generic/stat/main.libsonnet',
timeSeries: import './panels/generic/timeSeries/main.libsonnet',
table: import './panels/generic/table/main.libsonnet',
statusHistory: import './panels/generic/statusHistory/main.libsonnet',
},
network: {
timeSeries: import './panels/network/timeSeries/main.libsonnet',
},
system: {
stat: import './panels/system/stat/main.libsonnet',
table: import './panels/system/table/main.libsonnet',
statusHistory: import './panels/system/statusHistory/main.libsonnet',
},
cpu: {
stat: import './panels/cpu/stat/main.libsonnet',
timeSeries: import './panels/cpu/timeSeries/main.libsonnet',
},
memory: {
stat: import './panels/memory/stat/main.libsonnet',
timeSeries: import './panels/memory/timeSeries/main.libsonnet',
},
disk: {
timeSeries: import './panels/disk/timeSeries/main.libsonnet',
table: import './panels/disk/table/main.libsonnet',
stat: import './panels/disk/stat/main.libsonnet',
},
}
7 changes: 7 additions & 0 deletions common-lib/common/panels/cpu/stat/base.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
local g = import '../../../g.libsonnet';
local stat = g.panel.stat;
local base = import '../../generic/stat/base.libsonnet';

base {

}
21 changes: 21 additions & 0 deletions common-lib/common/panels/cpu/stat/count.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
local g = import '../../../g.libsonnet';
local generic = import '../../generic/stat/main.libsonnet';
local base = import './base.libsonnet';
local stat = g.panel.stat;

base {
new(
title='CPU count',
targets,
description=|||
CPU count is the number of processor cores or central processing units (CPUs) in a computer,
determining its processing capability and ability to handle tasks concurrently.
|||
):
super.new(title, targets, description),

stylize(allLayers=true):
(if allLayers then super.stylize() else {})
+ generic.info.stylize(allLayers=false),

}
5 changes: 5 additions & 0 deletions common-lib/common/panels/cpu/stat/main.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
base: import './base.libsonnet',
usage: import './usage.libsonnet',
count: import './count.libsonnet',
}
22 changes: 22 additions & 0 deletions common-lib/common/panels/cpu/stat/usage.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
local g = import '../../../g.libsonnet';
local generic = import '../../generic/stat/main.libsonnet';
local base = import './base.libsonnet';
local stat = g.panel.stat;

base {
new(
title='CPU usage',
targets,
description=|||
Total CPU utilization percent is a metric that indicates the overall level of central processing unit (CPU) usage in a computer system.
It represents the combined load placed on all CPU cores or processors.

For instance, if the total CPU utilization percent is 50%, it means that,
on average, half of the CPU's processing capacity is being used to execute tasks. A higher percentage indicates that the CPU is working more intensively, potentially leading to system slowdowns if it remains consistently high.
|||
):
super.new(title, targets, description),
stylize(allLayers=true):
(if allLayers then super.stylize() else {})
+ generic.percentage.stylize(allLayers=false),
}
6 changes: 6 additions & 0 deletions common-lib/common/panels/cpu/timeSeries/base.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
local g = import '../../../g.libsonnet';
local base = import '../../generic/timeSeries/base.libsonnet';

base {

}
5 changes: 5 additions & 0 deletions common-lib/common/panels/cpu/timeSeries/main.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
base: import './base.libsonnet',
utilization: import './utilization.libsonnet',
utilizationByMode: import './utilization_by_mode.libsonnet',
}
22 changes: 22 additions & 0 deletions common-lib/common/panels/cpu/timeSeries/utilization.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
local g = import '../../../g.libsonnet';
local generic = import '../../generic/timeSeries/main.libsonnet';
local base = import './base.libsonnet';
base {
new(
title='CPU usage',
targets,
description=|||
Total CPU utilization percent is a metric that indicates the overall level of central processing unit (CPU) usage in a computer system.
It represents the combined load placed on all CPU cores or processors.

For instance, if the total CPU utilization percent is 50%, it means that,
on average, half of the CPU's processing capacity is being used to execute tasks. A higher percentage indicates that the CPU is working more intensively, potentially leading to system slowdowns if it remains consistently high.
|||
):
super.new(title, targets, description)
+ self.stylize(),

stylize(allLayers=true):
(if allLayers then super.stylize() else {})
+ generic.percentage.stylize(allLayers=false),
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
local g = import '../../../g.libsonnet';
local base = import './base.libsonnet';
base {
new(
title='CPU usage by modes',
targets,
description='CPU usage by different modes.'
):
super.new(title, targets, description)
+ self.stylize(),

stylize(allLayers=true):
local timeSeries = g.panel.timeSeries;
local fieldOverride = g.panel.timeSeries.fieldOverride;

(if allLayers then super.stylize() else {})

+ timeSeries.standardOptions.withUnit('percent')
+ timeSeries.fieldConfig.defaults.custom.withFillOpacity(80)
+ timeSeries.fieldConfig.defaults.custom.withStacking({ mode: 'normal' })
+ timeSeries.standardOptions.withOverrides(
[
fieldOverride.byName.new('idle')
+ fieldOverride.byName.withPropertiesFromOptions(
timeSeries.standardOptions.color.withMode('fixed')
+ timeSeries.standardOptions.color.withFixedColor('light-blue'),
),
fieldOverride.byName.new('interrupt')
+ fieldOverride.byName.withPropertiesFromOptions(
timeSeries.standardOptions.color.withMode('fixed')
+ timeSeries.standardOptions.color.withFixedColor('light-purple'),
),
fieldOverride.byName.new('user')
+ fieldOverride.byName.withPropertiesFromOptions(
timeSeries.standardOptions.color.withMode('fixed')
+ timeSeries.standardOptions.color.withFixedColor('light-orange'),
),
fieldOverride.byRegexp.new('system|privileged')
+ fieldOverride.byRegexp.withPropertiesFromOptions(
timeSeries.standardOptions.color.withMode('fixed')
+ timeSeries.standardOptions.color.withFixedColor('light-red'),
),
]
),
}
6 changes: 6 additions & 0 deletions common-lib/common/panels/disk/stat/base.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
local g = import '../../../g.libsonnet';
local stat = g.panel.stat;
local base = import '../../generic/stat/base.libsonnet';

base {
}
4 changes: 4 additions & 0 deletions common-lib/common/panels/disk/stat/main.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
base: import './base.libsonnet',
total: import './total.libsonnet',
}
20 changes: 20 additions & 0 deletions common-lib/common/panels/disk/stat/total.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
local g = import '../../../g.libsonnet';
local generic = import '../../generic/stat/main.libsonnet';
local base = import './base.libsonnet';
local stat = g.panel.stat;

base {
new(
title,
targets,
description=''
):
super.new(title=title, targets=targets, description=description),

stylize(allLayers=true):

(if allLayers then super.stylize() else {})

+ generic.info.stylize(allLayers=false)
+ stat.standardOptions.withUnit('bytes'),
}
10 changes: 10 additions & 0 deletions common-lib/common/panels/disk/table/base.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
local g = import '../../../g.libsonnet';
local base = import '../../generic/table/base.libsonnet';
local table = g.panel.table;
local fieldOverride = g.panel.table.fieldOverride;
local custom = table.fieldConfig.defaults.custom;
local defaults = table.fieldConfig.defaults;
local options = table.options;
base {

}
4 changes: 4 additions & 0 deletions common-lib/common/panels/disk/table/main.libsonnet
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
base: import './base.libsonnet',
usage: import './usage.libsonnet',
}
Loading