Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v2: Ditch Query helpers. #17

Merged
merged 1 commit into from
Dec 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ jobs:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: '^1.16.2'
go-version: '^1.20'
- run: make test
- run: make build
- run: make codegendiff
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
.idea/*
.DS_store
14 changes: 1 addition & 13 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,9 @@ test:
$(GOBIN) test -v ./...

.PHONY: build
build: $(BINDIR)/codegen
build:
make -C tscli

$(BINDIR)/codegen: $(SCRIPTDIR)/codegen.go
@mkdir -p $(BINDIR)
GOOS=linux GOARCH=amd64 $(GOBIN) build -o $(BINDIR)/codegen $<

.PHONY: install
install: build
cp $(BINDIR)/tscli /usr/local/bin/
Expand All @@ -26,11 +22,3 @@ install: build
clean:
rm -rf $(BINDIR)

CODEGENFILE="set_aggregation_opts.go"
.PHONY: codegen
codegen:
$(BINDIR)/codegen -output ./tsplot/$(CODEGENFILE)

.PHONY: codegendiff
codegendiff:
diff ./tsplot/$(CODEGENFILE) <($(BINDIR)/codegen -stdout)
82 changes: 21 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,84 +2,44 @@
[![Go Report Card](https://goreportcard.com/badge/github.com/bitly/tsplot)](https://goreportcard.com/report/github.com/bitly/tsplot)
[![Go Reference](https://pkg.go.dev/badge/github.com/bitly/tsplot.svg)](https://pkg.go.dev/github.com/bitly/tsplot)

This package provides a method of querying for raw time series data from the GCM APIs and additionally plotting that data for use in other applications.

This came to be due to what we consider a small limitation in the Google APIs which require us to re-draw graphs to include them in other applications such as
Slack bots. There is no facility in the Google API that provides a PNG of already graphed data.

## Authentication
This package makes no effort to assist in authentication to the Google APIs.
Instead, it will expect the caller to supply an authenticated client.

More information on authentication can be found in the official [Google Cloud documentation](https://cloud.google.com/docs/authentication).

## Query
tsplot helps to facilitate easy querying of the Google Cloud Monitoring API for time series matching the supplied criteria.
In addition it provides methods of overriding certain aspects of the query.

For example, the following code snippet will return a single time series for the following metric descriptor: `custom.googleapis.com/opencensus/fishnet/queuereader_fishnet/messages_total`.
```
func main() {

... snip ...
... snip ...

start := time.Now().Add(-1 * time.Hour)
end := time.Now()
mq := &tsplot.NewMetricQuery(
"bitly-gcp-prod", // GCP project
"custom.googleapis.com/opencensus/fishent/queuereader_fishnet/messages_total", // metric descriptor
&start, // start of time window
&end, // end of time window
)

// disable cross series reducer (MEAN reduction is default)
query.Set_REDUCE_NONE()

// set different alignment window. (Default is 1 minute)
query.SetAlignmentPeriod(time.Minute * 2)

tsi, err := mq.PerformWithClient(client) // client is provided by user
if err != nil {
fmt.Printf("error performing query: %v\n", err)
}
}
```

## Plotting
To plot the data, tsplot leverages the open source package [gonum/plot](github.com/gonum/plot) to create a graph and plot the data for a given time series.

The example below creates a new graph containing a singular time series, plots it, and saves the resulting plot to disk.
```
func main() {

... snip ...

ts := tsplot.TimeSeries{}

// optionally iterate over returned time series
timeSeries, _ := tsi.Next()
ts[metric] = ts.GetPoints()

// create the plot with some formatting options
p, err := ts.Plot([]tsplot.PlotOption{
tsplot.WithXAxisName("UTC"),
tsplot.WIthGrid(colornames.Darkgrey),
tsplot.WithTitle(metric)}...)

if err != nil {
return err
// create new request
request := monitoringpb.ListTimeSeriesRequest{
Name: fmt.Sprintf("projects/%s", project),
Filter: query,
Interval: &monitoringpb.TimeInterval{
EndTime: timestamppb.New(et),
StartTime: timestamppb.New(st),
},
Aggregation: nil,
SecondaryAggregation: nil,
View: monitoringpb.ListTimeSeriesRequest_FULL,
}

// optionally save the plot to disk
p.Save(8*vg.Inch, 4*vg.Inch, "./my-graph.png")

// execute the request and get the response from Google APIs
tsi := GoogleCloudMonitoringClient.ListTimeSeries(context.Background(), request)

// Create the plot from the GAPI TimeSeries
plot, _ := tsplot.NewPlotFromTimeSeriesIterator(tsi, "", nil)

// Save the new plot to disk.
plot.Save(8*vg.Inch, 4*vg.Inch, "my_plot.png")
}
```

### Example generated graphs:
Query across multiple time series with mean reducer:
![graph1](sample/1.png)

### Graph Color Scheme
I'm not a UX designer, but I have selected colors that I find higher contrast
and easier to see. I am basing this completely off my colorblindness which is
unique to me. Improvements to the color palette used are welcome.
24 changes: 23 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/bitly/tsplot

go 1.14
go 1.20

require (
cloud.google.com/go v0.80.0
Expand All @@ -11,3 +11,25 @@ require (
google.golang.org/genproto v0.0.0-20210401141331-865547bb08e2
google.golang.org/protobuf v1.26.0
)

require (
github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af // indirect
github.com/fogleman/gg v1.3.0 // indirect
github.com/go-fonts/liberation v0.1.1 // indirect
github.com/go-latex/latex v0.0.0-20210118124228-b3d85cf34e07 // indirect
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect
github.com/golang/protobuf v1.5.1 // indirect
github.com/google/go-cmp v0.5.5 // indirect
github.com/googleapis/gax-go/v2 v2.0.5 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/phpdave11/gofpdf v1.4.2 // indirect
github.com/spf13/pflag v1.0.5 // indirect
go.opencensus.io v0.23.0 // indirect
golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4 // indirect
golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84 // indirect
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4 // indirect
golang.org/x/text v0.3.5 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/grpc v1.36.1 // indirect
)
1 change: 0 additions & 1 deletion go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,6 @@ golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8T
gonum.org/v1/gonum v0.0.0-20180816165407-929014505bf4/go.mod h1:Y+Yx5eoAFn32cQvJDxZx5Dpnq+c3wtXuadVZAcxbbBo=
gonum.org/v1/gonum v0.8.2 h1:CCXrcPKiGGotvnN6jfUsKk4rRqm7q09/YbKb5xCEvtM=
gonum.org/v1/gonum v0.8.2/go.mod h1:oe/vMfY3deqTw+1EZJhuvEW2iwGF1bW9wwu7XCu0+v0=
gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0 h1:OE9mWmgKkjJyEmDAAtGMPjXu+YNeGvK9VTSHY6+Qihc=
gonum.org/v1/netlib v0.0.0-20190313105609-8cb42192e0e0/go.mod h1:wa6Ws7BG/ESfp6dHfk7C6KdzKA7wR7u/rKwOGE66zvw=
gonum.org/v1/plot v0.0.0-20190515093506-e2840ee46a6b/go.mod h1:Wt8AAjI+ypCyYX3nZBvf6cAIx93T+c/OS2HFAYskSZc=
gonum.org/v1/plot v0.9.0 h1:3sEo36Uopv1/SA/dMFFaxXoL5XyikJ9Sf2Vll/k6+2E=
Expand Down
67 changes: 0 additions & 67 deletions scripts/codegen.go

This file was deleted.

2 changes: 1 addition & 1 deletion tscli/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ default: build
.PHONY: build
build:
@mkdir -p $(BINDIR)
GOOS=linux GOARCH=amd64 $(GOBIN) build -o $(BINDIR)/tscli
$(GOBIN) build -o $(BINDIR)/tscli
Loading
Loading