Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added logging of Slurm subprocess failures and testing for all parsing functions #43

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8420a24
Corrected function name in cpus_test.go
Rovanion Feb 12, 2021
a04f2a2
Split tests into unit and system tests
Rovanion Feb 12, 2021
e7064fd
Corrected formatting of the project
Rovanion Feb 18, 2021
e5d513b
Add prometheus-slurm-exporter to .gitignore
Rovanion Feb 22, 2021
66472ef
Added Logging library
Rovanion Feb 22, 2021
38aae50
Made accounts.go print error messages from subprocesses
Rovanion Feb 22, 2021
af6975f
Added two logging functions for use in utility functions
Rovanion Feb 24, 2021
403a39f
Extracted the unix command execution from accounts.go to a generic su…
Rovanion Feb 24, 2021
706748f
Refactored cpus to use subprocess.go
Rovanion Feb 25, 2021
8644bba
Refactored gpus.go to use subprocess.go and also added unit and syste…
Rovanion Feb 25, 2021
2b46da1
Refactored nodes.go to use subprocess.go
Rovanion Feb 25, 2021
95c6ffb
Refactored partitions.go to use subprocess.go
Rovanion Feb 26, 2021
fe6be80
Added system and unit tests for partition data gathering
Rovanion Feb 26, 2021
2fd60b2
Refactored queue.go to use subprocess.go
Rovanion Feb 26, 2021
f1b1d22
Added real unit tests for queue.go
Rovanion Feb 26, 2021
260c772
Refactored scheduler.go to use subprocess.go
Rovanion Feb 26, 2021
e851819
Added actual unit tests for scheduler.go
Rovanion Feb 26, 2021
00a4cbc
Refactored sshare.go to use subprocess.go
Rovanion Mar 1, 2021
13c7526
Added tests for sshare.go
Rovanion Mar 1, 2021
bb630bd
Refactored users.go to use subprocess.go
Rovanion Mar 1, 2021
d1b68e6
Added tests for users.go
Rovanion Mar 1, 2021
fbda610
Clearify an argument name in accounts.go
Rovanion Mar 1, 2021
3574e0e
Added tests for accounts.go
Rovanion Mar 1, 2021
bcc30db
Corrected function name in nodes_system_test.go
Rovanion Mar 1, 2021
ca0012f
Corrected imports in system tests for partitions, users and sshare
Rovanion Mar 1, 2021
ec689e7
Corrected function name in users_system_test.go
Rovanion Mar 1, 2021
a755555
Implemented an actual unit tests for cpus.go's ParseCPUsMetrics
Rovanion Mar 1, 2021
dca9b63
Wrote actual tests for nodes.go
Rovanion Mar 2, 2021
9a3570b
Merge remote-tracking branch 'origin/master' into log-slurm-failure-o…
Rovanion Aug 4, 2021
9afe3af
Refactored node.go to use subprocess.go
Rovanion Aug 4, 2021
611df5a
Revert e7064fd, undoing style fixes
Rovanion Aug 4, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Refactored node.go to use subprocess.go
  • Loading branch information
Rovanion committed Aug 4, 2021
commit 9afe3af18431ab4788a98905e7602dd765759f3e
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
@@ -5,4 +5,5 @@ go 1.12
require (
github.com/prometheus/client_golang v1.2.1
github.com/prometheus/common v0.7.0
github.com/stretchr/testify v1.3.0 // indirect
)
19 changes: 3 additions & 16 deletions node.go
Original file line number Diff line number Diff line change
@@ -16,8 +16,6 @@ along with this program. If not, see <http://www.gnu.org/licenses/>. */
package main

import (
"log"
"os/exec"
"sort"
"strconv"
"strings"
@@ -36,8 +34,8 @@ type NodeMetrics struct {
nodeStatus string
}

func NodeGetMetrics() map[string]*NodeMetrics {
return ParseNodeMetrics(NodeData())
func GetNodeMetrics() map[string]*NodeMetrics {
return ParseNodeMetrics(Subprocess("sinfo", "-h", "-N", "-O", "NodeList,AllocMem,Memory,CPUsState,StateLong"))
}

// ParseNodeMetrics takes the output of sinfo with node data
@@ -79,17 +77,6 @@ func ParseNodeMetrics(input []byte) map[string]*NodeMetrics {
return nodes
}

// NodeData executes the sinfo command to get data for each node
// It returns the output of the sinfo command
func NodeData() []byte {
cmd := exec.Command("sinfo", "-h", "-N", "-O", "NodeList,AllocMem,Memory,CPUsState,StateLong")
out, err := cmd.Output()
if err != nil {
log.Fatal(err)
}
return out
}

type NodeCollector struct {
cpuAlloc *prometheus.Desc
cpuIdle *prometheus.Desc
@@ -125,7 +112,7 @@ func (nc *NodeCollector) Describe(ch chan<- *prometheus.Desc) {
}

func (nc *NodeCollector) Collect(ch chan<- prometheus.Metric) {
nodes := NodeGetMetrics()
nodes := GetNodeMetrics()
for node := range nodes {
ch <- prometheus.MustNewConstMetric(nc.cpuAlloc, prometheus.GaugeValue, float64(nodes[node].cpuAlloc), node, nodes[node].nodeStatus)
ch <- prometheus.MustNewConstMetric(nc.cpuIdle, prometheus.GaugeValue, float64(nodes[node].cpuIdle), node, nodes[node].nodeStatus)
5 changes: 3 additions & 2 deletions node_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/* Copyright 2021 Chris Read
// +build unit
/* Copyright 2021 Chris Read, Rovanion Luckey

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -45,7 +46,7 @@ func TestNodeMetrics(t *testing.T) {
t.Fatalf("Can not open test data: %v", err)
}
metrics := ParseNodeMetrics(data)
t.Logf("%+v", metrics)
// t.Logf("%+v", metrics)

assert.Contains(t, metrics, "b001")
assert.Equal(t, uint64(327680), metrics["b001"].memAlloc)
1 change: 1 addition & 0 deletions nodes.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/* Copyright 2017 Victor Penso, Matteo Dessalvi
Copyright 2021 Rovanion Luckey

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by