-
Notifications
You must be signed in to change notification settings - Fork 487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add grafanacloud-specific install script #291
Changes from all commits
d851051
b7e2964
bcd3d79
033a1ed
9b964da
c4e243f
2960322
9d826a8
e85cd41
cf4c956
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
// Package grafanacloud provides an interface to the Grafana Cloud API. | ||
package grafanacloud | ||
|
||
import ( | ||
"context" | ||
"fmt" | ||
"net/http" | ||
"strings" | ||
|
||
"gopkg.in/yaml.v2" | ||
) | ||
|
||
const integrationsAPIURL = "https://integrations-api.grafana.net" | ||
|
||
// Client is a grafanacloud API client. | ||
type Client struct { | ||
c *http.Client | ||
apiKey string | ||
} | ||
|
||
// NewClient creates a new Grafana Cloud client. All requests made will be | ||
// performed using the provided http.Client c. If c is nil, the default | ||
// http client will be used instead. | ||
// | ||
// apiKey will be used to authenticate against the API. | ||
func NewClient(c *http.Client, apiKey string) *Client { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What do you think about adding There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the client code is too low level for environment variable logic. The caller can be responsible for retrieving environment variables, if necessary. (That being said, I don't really like environment variables for configuration, and I think having the flag here is enough given this is a hidden command we don't want to document) |
||
if c == nil { | ||
c = http.DefaultClient | ||
} | ||
return &Client{c: c, apiKey: apiKey} | ||
} | ||
|
||
// AgentConfig generates a Grafana Cloud Agent config from the given stack. | ||
// The config is returned as a string in YAML form. | ||
func (c *Client) AgentConfig(ctx context.Context, stackID string) (string, error) { | ||
req, err := http.NewRequestWithContext( | ||
ctx, "GET", | ||
fmt.Sprintf("%s/stacks/%s/agent_config", integrationsAPIURL, stackID), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What do you think about using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does that endpoint exist? I get a 404 when I try it. Given the Agent will be responsible for stitching together pieces of information in the future, I think it's fine for the logic to exist here. Plus it was faster for me to add client-side than to put it on your backlog with Evan. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
No, we would have to add it.
I wonder how much logic the Agent must really have for the config. I'm afraid that if we go too far we block users from using their config management of choice. If the config consists of plain files users can manipulate them as they like. However, if they must use another tool it becomes more complicated. I'm probably missing a lot here. Do we have a document on future plans? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Users will always be able to use whatever software they want to configure the Agent. This PR is just for the turn-key get-started install script on Grafana Cloud; we will never have a hard dependency on building a tool that must be used for configuring the Agent. |
||
nil, | ||
) | ||
if err != nil { | ||
return "", fmt.Errorf("failed to generate request: %w", err) | ||
} | ||
req.Header.Add("Authorization", "Bearer "+c.apiKey) | ||
|
||
resp, err := c.c.Do(req) | ||
if err != nil { | ||
return "", fmt.Errorf("failed to make request: %w", err) | ||
} | ||
defer resp.Body.Close() | ||
|
||
if resp.StatusCode != 200 { | ||
return "", fmt.Errorf("unexpected status code %d", resp.StatusCode) | ||
} | ||
|
||
// Even though the API returns json, we'll parse it as YAML here so we can | ||
// re-encode it with the same order it was decoded in. | ||
payload := struct { | ||
Status string `yaml:"status"` | ||
Data yaml.MapSlice `yaml:"data"` | ||
Error string `yaml:"error"` | ||
}{} | ||
|
||
dec := yaml.NewDecoder(resp.Body) | ||
if err := dec.Decode(&payload); err != nil { | ||
return "", fmt.Errorf("failed to read response: %w", err) | ||
} | ||
|
||
if payload.Status != "success" { | ||
return "", fmt.Errorf("request was not successful: %s", payload.Error) | ||
} | ||
|
||
// Convert the data to YAML | ||
var sb strings.Builder | ||
if err := yaml.NewEncoder(&sb).Encode(payload.Data); err != nil { | ||
return "", fmt.Errorf("failed to generate YAML config: %w", err) | ||
} | ||
|
||
return sb.String(), nil | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,98 @@ | ||
package grafanacloud | ||
|
||
import ( | ||
"context" | ||
"crypto/tls" | ||
"fmt" | ||
"net" | ||
"net/http" | ||
"net/http/httptest" | ||
"testing" | ||
|
||
"github.com/stretchr/testify/assert" | ||
"github.com/stretchr/testify/require" | ||
) | ||
|
||
const ( | ||
testSecret = "secret-key" | ||
testStackID = "12345" | ||
) | ||
|
||
func TestClient_AgentConfig(t *testing.T) { | ||
httpClient := testClient(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
assert.Equal(t, "/stacks/"+testStackID+"/agent_config", r.URL.Path) | ||
assert.Equal(t, "Bearer "+testSecret, r.Header.Get("Authorization")) | ||
|
||
_, err := w.Write([]byte(`{ | ||
"status": "success", | ||
"data": { | ||
"server": { | ||
"http_listen_port": 12345 | ||
}, | ||
"integrations": { | ||
"agent": { | ||
"enabled": true | ||
} | ||
} | ||
} | ||
}`)) | ||
assert.NoError(t, err) | ||
})) | ||
|
||
cli := NewClient(httpClient, testSecret) | ||
cfg, err := cli.AgentConfig(context.Background(), testStackID) | ||
require.NoError(t, err) | ||
fmt.Println(cfg) | ||
|
||
expect := ` | ||
server: | ||
http_listen_port: 12345 | ||
integrations: | ||
agent: | ||
enabled: true | ||
` | ||
|
||
require.YAMLEq(t, expect, cfg) | ||
} | ||
|
||
func TestClient_AgentConfig_Error(t *testing.T) { | ||
httpClient := testClient(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
w.WriteHeader(http.StatusNotFound) | ||
})) | ||
|
||
cli := NewClient(httpClient, testSecret) | ||
_, err := cli.AgentConfig(context.Background(), testStackID) | ||
require.Error(t, err, "unexpected status code 404") | ||
} | ||
|
||
func TestClient_AgentConfig_ErrorMessage(t *testing.T) { | ||
httpClient := testClient(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { | ||
_, err := w.Write([]byte(`{ | ||
"status": "error", | ||
"error": "Something went wrong" | ||
}`)) | ||
assert.NoError(t, err) | ||
})) | ||
|
||
cli := NewClient(httpClient, testSecret) | ||
_, err := cli.AgentConfig(context.Background(), testStackID) | ||
require.Error(t, err, "request was not successful: Something went wrong") | ||
} | ||
|
||
func testClient(t *testing.T, handler http.HandlerFunc) *http.Client { | ||
h := httptest.NewTLSServer(handler) | ||
t.Cleanup(func() { | ||
h.Close() | ||
}) | ||
|
||
return &http.Client{ | ||
Transport: &http.Transport{ | ||
TLSClientConfig: &tls.Config{ | ||
InsecureSkipVerify: true, | ||
}, | ||
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) { | ||
return net.Dial(network, h.Listener.Addr().String()) | ||
}, | ||
}, | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,118 @@ | ||
#!/usr/bin/env sh | ||
# grafanacloud-install.sh installs the Grafana Cloud Agent on supported x86_64 | ||
# Linux systems for Grafana Cloud users. Those who aren't users of Grafana Cloud | ||
# or need to install the Agent on a different architecture or platform should | ||
# try another installation method. | ||
# | ||
# grafanacloud-install.sh has a hard dependency on being run on a supported | ||
# Linux system. Currently only systems that can install deb or rpm packages | ||
# are supported. The target system will try to be detected, but if it cannot, | ||
# PACKAGE_SYSTEM can be passed as an environment variable with either rpm or | ||
# deb. | ||
set -eu | ||
|
||
log() { | ||
echo "$@" >&2 | ||
} | ||
|
||
fatal() { | ||
log "$@" | ||
exit 1 | ||
} | ||
|
||
# | ||
# REQUIRED environment variables. | ||
# | ||
GCLOUD_STACK_ID=${GCLOUD_STACK_ID:=} # Stack ID where integrations are installed | ||
GCLOUD_API_KEY=${GCLOUD_API_KEY:=} # API key to communicate to the integrations API | ||
|
||
[ -z "$GCLOUD_STACK_ID" ] && fatal "Required environment variable \$GCLOUD_STACK_ID not set." | ||
[ -z "$GCLOUD_API_KEY" ] && fatal "Required environment variable \$GCLOUD_API_KEY not set." | ||
|
||
# | ||
# OPTIONAL environment variables. | ||
# | ||
|
||
# Package system to install the Agent with. If not empty, MUST be either rpm or | ||
# deb. If empty, the script will try to detect the host OS and the appropriate | ||
# package system to use. | ||
PACKAGE_SYSTEM=${PACKAGE_SYSTEM:=} | ||
|
||
# | ||
# Global constants. | ||
# | ||
RELEASE_VERSION="0.9.0" | ||
|
||
RELEASE_URL="https://github.com/grafana/agent/releases/download/v${RELEASE_VERSION}" | ||
DEB_URL="${RELEASE_URL}/grafana-agent-${RELEASE_VERSION}-1.x86_64.deb" | ||
RPM_URL="${RELEASE_URL}/grafana-agent-${RELEASE_VERSION}-1.x86_64.rpm" | ||
|
||
main() { | ||
if [ -z "$PACKAGE_SYSTEM" ]; then | ||
PACKAGE_SYSTEM=$(detect_package_system) | ||
fi | ||
log "--- Using package system $PACKAGE_SYSTEM. Downloading and installing package" | ||
|
||
case "$PACKAGE_SYSTEM" in | ||
deb) | ||
install_deb | ||
;; | ||
rpm) | ||
install_rpm | ||
;; | ||
*) | ||
fatal "Unsupported PACKAGE_SYSTEM value $PACKAGE_SYSTEM. Must be either rpm or deb". | ||
;; | ||
esac | ||
|
||
log '--- Retrieving config and placing in /etc/grafana-agent.yaml' | ||
retrieve_config | sudo tee /etc/grafana-agent.yaml | ||
|
||
log '--- Enabling and starting grafana-agent.service' | ||
sudo systemctl enable grafana-agent.service | ||
sudo systemctl start grafana-agent.service | ||
|
||
# Add some empty newlines to give some visual whitespace before printing the | ||
# success message. | ||
log '' | ||
log '' | ||
log 'Grafana Cloud Agent is now running! To check the status of your Agent, run:' | ||
log ' sudo systemctl status grafana-agent.service' | ||
} | ||
|
||
# detect_package_system tries to detect the host distribution to determine if | ||
# deb or rpm should be used for installing the Agent. Prints out either "deb" | ||
# or "rpm". Calls fatal if the host OS is not supported. | ||
detect_package_system() { | ||
command -v dpkg >/dev/null 2>&1 && { echo "deb"; return; } | ||
command -v rpm >/dev/null 2>&1 && { echo "rpm"; return; } | ||
|
||
case "$(uname)" in | ||
Darwin) | ||
fatal 'macOS not supported' | ||
;; | ||
*) | ||
fatal "Unknown unsupported OS: $(uname)" | ||
;; | ||
esac | ||
} | ||
|
||
# install_deb downloads and installs the deb package of the Grafana Cloud Agent. | ||
install_deb() { | ||
curl -sL "${DEB_URL}" -o /tmp/grafana-agent.deb | ||
sudo dpkg -i /tmp/grafana-agent.deb | ||
rm /tmp/grafana-agent.deb | ||
} | ||
|
||
# install_rpm downloads and installs the deb package of the Grafana Cloud Agent. | ||
install_rpm() { | ||
sudo rpm --reinstall "${RPM_URL}" | ||
} | ||
|
||
# retrieve_config downloads the config file for the Agent and prints out its | ||
# contents to stdout. | ||
retrieve_config() { | ||
grafana-agentctl cloud-config -u "${GCLOUD_STACK_ID}" -p "${GCLOUD_API_KEY}" || fatal 'Failed to retrieve config' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I probably miss something but why not use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The implementation of It may turn out that we even remove the package later on, but I suspect it's also where the C&C client-side code will be written. |
||
} | ||
|
||
main |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is out of scope but we really should start consolidating our GCom clients 🙂