Add grafanacloud-specific install script #291

rfratto · 2020-12-15T19:49:42Z

PR Description

Adds a Grafana Cloud-specific install script that does the following:

Downloads the rpm or deb Agent package and installs it
Uses a new hidden agentctl command to generate the config file using an internal Grafana Cloud API
Enables and starts the service.

This script can be used by setup instructions on Grafana Cloud.

Which issue(s) this PR fixes

Notes to the Reviewer

This currently won't work completely as grafana-agentctl isn't installed by the 0.9.0 binaries. To reproduce locally, install agentctl as grafana-agentctl. A 0.9.1 release is needed for this script to work fully on its own.

PR Checklist

CHANGELOG updated (not relevant)
Documentation added (not relevant)
Tests updated

rfratto · 2020-12-15T19:50:18Z

Tagging @jdbaldry for just the review of the bash script in case there's any insights there, @hoenn for the grafanacloud client code.

hoenn

lgtm for gcloud, 👍 /👎 works with your stack?

cmd/agentctl/main.go

Co-authored-by: Evan <10603766+hoenn@users.noreply.github.com>

rfratto · 2020-12-15T20:00:29Z

yeah, it generated a sane config for my stack

jdbaldry

Bash script looks good to me, one change requested to fix the required variable messages but approved to avoid blocking :)

One other nit is that there is some trailing whitespace throughout the script which has been highlighted by my editor.

production/grafanacloud-install.sh

jeschkies · 2020-12-16T09:16:20Z

pkg/client/grafanacloud/client.go

@@ -0,0 +1,80 @@
+// Package grafanacloud provides an interface to the Grafana Cloud API.
+package grafanacloud


This is out of scope but we really should start consolidating our GCom clients 🙂

jeschkies · 2020-12-16T09:17:15Z

pkg/client/grafanacloud/client.go

+// http client will be used instead.
+//
+// apiKey will be used to authenticate against the API.
+func NewClient(c *http.Client, apiKey string) *Client {


What do you think about adding NewClientFromEnv that assumes a configured GCLOUD_API_KEY?

I think the client code is too low level for environment variable logic. The caller can be responsible for retrieving environment variables, if necessary.

(That being said, I don't really like environment variables for configuration, and I think having the flag here is enough given this is a hidden command we don't want to document)

jeschkies · 2020-12-16T09:18:44Z

pkg/client/grafanacloud/client.go

+func (c *Client) AgentConfig(ctx context.Context, stackID string) (string, error) {
+	req, err := http.NewRequestWithContext(
+		ctx, "GET",
+		fmt.Sprintf("%s/stacks/%s/agent_config", integrationsAPIURL, stackID),


What do you think about using %s/stacks/%s/agent_config.yaml instead? What logic should live in the client?

Does that endpoint exist? I get a 404 when I try it.

Given the Agent will be responsible for stitching together pieces of information in the future, I think it's fine for the logic to exist here. Plus it was faster for me to add client-side than to put it on your backlog with Evan.

Does that endpoint exist? I get a 404 when I try it.

No, we would have to add it.

Given the Agent will be responsible for stitching together pieces of information in the future,

I wonder how much logic the Agent must really have for the config. I'm afraid that if we go too far we block users from using their config management of choice. If the config consists of plain files users can manipulate them as they like. However, if they must use another tool it becomes more complicated. I'm probably missing a lot here. Do we have a document on future plans?

Users will always be able to use whatever software they want to configure the Agent. This PR is just for the turn-key get-started install script on Grafana Cloud; we will never have a hard dependency on building a tool that must be used for configuring the Agent.

production/grafanacloud-install.sh

jeschkies · 2020-12-16T09:27:01Z

production/grafanacloud-install.sh

+# retrieve_config downloads the config file for the Agent and prints out its
+# contents to stdout.
+retrieve_config() {
+  grafana-agentctl cloud-config -u "${GCLOUD_STACK_ID}" -p "${GCLOUD_API_KEY}" || fatal 'Failed to retrieve config'


I probably miss something but why not use curl ${API_ENDPOINT}/stacks/${GCLOUD_STACK_ID}/agent.yaml instead? The GCom client seems to be overhead unless we'll add more logic I am not aware of.

The implementation of cloud-config will probably change over time when integrations are decoupled from the API implementation. I'm comfortable maintaining a tiny package on the client side that's responsible for stitching together the final config file (which is where I think the logic should live anyway - the integrations API should, ideally, just give me config for integrations and remote_write info, and the Agent should be responsible for knowing how it should be configured outside of that scope).

It may turn out that we even remove the package later on, but I suspect it's also where the C&C client-side code will be written.

Co-authored-by: Jack Baldry <jack.baldry@grafana.com>

Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>

rfratto · 2020-12-16T14:39:18Z

Tested on both rpm- and deb-based distros and verified this script is working as intended. I'm going to merge this now given it has two approvals, but @jeschkies feel free to reach out offline if we need to discuss more about this, I can backpedal later if necessary.

* grafanacloud-specific install script * remove empty comment * Update cmd/agentctl/main.go Co-authored-by: Evan <10603766+hoenn@users.noreply.github.com> * appease the linter * Update production/grafanacloud-install.sh Co-authored-by: Jack Baldry <jack.baldry@grafana.com> * Update production/grafanacloud-install.sh Co-authored-by: Jack Baldry <jack.baldry@grafana.com> * Update production/grafanacloud-install.sh Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz> * review feedback * agentctl: change to consistent command function naming * print success message after installing/starting agent service Co-authored-by: Evan <10603766+hoenn@users.noreply.github.com> Co-authored-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>

grafanacloud-specific install script

d851051

rfratto requested review from jdbaldry, hoenn and a team December 15, 2020 19:49

remove empty comment

b7e2964

hoenn approved these changes Dec 15, 2020

View reviewed changes

cmd/agentctl/main.go Outdated Show resolved Hide resolved

Update cmd/agentctl/main.go

bcd3d79

Co-authored-by: Evan <10603766+hoenn@users.noreply.github.com>

appease the linter

033a1ed

jdbaldry approved these changes Dec 16, 2020

View reviewed changes

production/grafanacloud-install.sh Outdated Show resolved Hide resolved

production/grafanacloud-install.sh Outdated Show resolved Hide resolved

production/grafanacloud-install.sh Outdated Show resolved Hide resolved

jeschkies reviewed Dec 16, 2020

View reviewed changes

rfratto and others added 6 commits December 16, 2020 05:00

Update production/grafanacloud-install.sh

9b964da

Co-authored-by: Jack Baldry <jack.baldry@grafana.com>

Update production/grafanacloud-install.sh

c4e243f

Co-authored-by: Jack Baldry <jack.baldry@grafana.com>

Update production/grafanacloud-install.sh

2960322

Co-authored-by: Karsten Jeschkies <k@jeschkies.xyz>

review feedback

9d826a8

agentctl: change to consistent command function naming

e85cd41

print success message after installing/starting agent service

cf4c956

rfratto merged commit 5d9efdb into grafana:master Dec 16, 2020

rfratto deleted the grafanacloud-install-script branch December 16, 2020 14:39

mattdurham mentioned this pull request Sep 7, 2021

crow doc rfratto/agent#8

Closed

3 tasks

github-actions bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Apr 22, 2024

github-actions bot locked as resolved and limited conversation to collaborators Apr 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add grafanacloud-specific install script #291

Add grafanacloud-specific install script #291

rfratto commented Dec 15, 2020

rfratto commented Dec 15, 2020

hoenn left a comment

rfratto commented Dec 15, 2020

jdbaldry left a comment •

edited

Loading

jeschkies Dec 16, 2020

jeschkies Dec 16, 2020

rfratto Dec 16, 2020

jeschkies Dec 16, 2020

rfratto Dec 16, 2020

jeschkies Dec 16, 2020

rfratto Dec 16, 2020

jeschkies Dec 16, 2020

rfratto Dec 16, 2020 •

edited

Loading

rfratto commented Dec 16, 2020

		@@ -0,0 +1,80 @@
		// Package grafanacloud provides an interface to the Grafana Cloud API.
		package grafanacloud

Add grafanacloud-specific install script #291

Add grafanacloud-specific install script #291

Conversation

rfratto commented Dec 15, 2020

PR Description

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

rfratto commented Dec 15, 2020

hoenn left a comment

Choose a reason for hiding this comment

rfratto commented Dec 15, 2020

jdbaldry left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rfratto Dec 16, 2020 • edited Loading

Choose a reason for hiding this comment

rfratto commented Dec 16, 2020

jdbaldry left a comment •

edited

Loading

rfratto Dec 16, 2020 •

edited

Loading