Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: revise getting started guide #5939

Merged
merged 12 commits into from
Apr 28, 2022
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 132 additions & 5 deletions docs/sources/getting-started/_index.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,141 @@
---
title: Getting started
weight: 300
description: "This guide assists the reader to create and use a simple Loki cluster for testing and evaluation purposes."
aliases:
- /docs/loki/latest/getting-started/get-logs-into-loki/
---

# Getting started with Grafana Loki

> **Note:** You can use [Grafana Cloud](https://grafana.com/products/cloud/features/#cloud-logs) to avoid installing, maintaining, and scaling your own instance of Grafana Loki. The free forever plan includes 50GB of free logs. [Create an account to get started](https://grafana.com/auth/sign-up/create-user?pg=docs-loki&plcmt=in-text).

1. [Getting Logs Into Loki](get-logs-into-loki/)
1. [Grafana](grafana/)
1. [LogCLI](logcli/)
1. [Labels](labels/)
1. [Troubleshooting](troubleshooting/)
This guide assists the reader to create and use a simple Loki cluster.
The cluster is intended for testing, development, and evaluation;
it will not meet most production requirements.

The test environment runs the [flog](https://github.com/mingrammer/flog) app to generate log lines.
Promtail is the test environment's agent (or client) that captures the log lines and pushes them to the Loki cluster through a gateway.
In a typical environment, the log-generating app and the agent run together, but in locations distinct from the Loki cluster. This guide runs each piece of the test environment locally, in Docker containers.

Grafana provides a way to pose queries against the logs stored in Loki and visualize query results.

![Simple scalable deployment test environment](simple-scalable-test-environment.png)

The test environment uses Docker compose to instantiate these parts, each in its own container:

- One [single scalable deployment](../fundamentals/architecture/deployment-modes/) mode **Loki** instance has:
- One Loki read component
- One Loki write component
- **Minio** is Loki's storage back end in the test environment.
- The **gateway** receives requests and redirects them to the appropriate container based on the request's URL.
- **Flog** generates log lines.
- **Promtail** scrapes the log lines from flog, and pushes them to Loki through the gateway.
- **Grafana** provides visualization of the log lines captured within Loki.

## Prerequisites

- [Docker](https://docs.docker.com/install)
- [Docker Compose](https://docs.docker.com/compose/install)

## Obtain the test environment

1. Create a directory called `evaluate-loki` for the test environment. Make `evaluate-loki` your current working directory:
```bash
mkdir evaluate-loki
cd evaluate-loki
```
1. Download `loki-config.yaml`, `promtail-local-config.yaml`, and `docker-compose.yaml`:

```bash
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/promtail-local-config.yaml -O promtail-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml
```

## Deploy the test environment

All shell commands are issued from the `evaluate-loki` directory.

1. With `evaluate-loki` as the current working directory, deploy the test environment using `docker-compose`:
```bash
docker-compose up
KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved
KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved
```
1. (Optional) Verify that the Loki cluster is up and running. The read component returns `ready` when you point a web browser at http://localhost:3101/ready. The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready.
The write component returns `ready` when you point a web browser at http://localhost:3102/ready. The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready.

## Use Grafana and the test environment

Use [Grafana](https://grafana.com/docs/grafana/latest/) to query and observe the log lines captured in the Loki cluster by navigating a browser to http://localhost:3000.
The Grafana instance has Loki configured as a [datasource](https://grafana.com/docs/grafana/latest/datasources/loki/).

Click on the Grafana instance's [Explore](https://grafana.com/docs/grafana/latest/explore/) icon to bring up the explore pane.

Use the Explore dropdown menu to choose the Loki datasource and bring up the Loki query browser.

Try some queries.
Enter your query into the **Log browser** box, and click on the blue **Run query** button.
KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved

To see all the log lines that flog has generated:
```
{container="evaluate-loki_flog_1"}
```

The flog app will generate log lines for invented HTTP requests.
To see all `GET` log lines, enter the query:

```
{container="evaluate-loki_flog_1"} |= "GET"
```
For `POST` methods:
```
{container="evaluate-loki_flog_1"} |= "POST"
```

To see every log line with the value of 401:
KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved
```
{container="evaluate-loki_flog_1"} |= "401"
KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved

```
To see every log line other than those that contain the value 401:
```
{container="evaluate-loki_flog_1"} != "401"
```

KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved
## Stop and clean up the test environment

To break down the test environment:

- Close the Grafana browser window

- Stop and remove all the Docker containers. With `evaluate-loki` as the current working directory:
```bash
docker-compose down
```

## Modifying the flog app output
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm I think this subsection seems out-of-place/looks unnecessary. WDYT?

Copy link
Contributor Author

@KMiller-Grafana KMiller-Grafana Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see what others say in their reviews. I wanted this effort to be a guide, not a tutorial. Part of making a guide is providing paths for extending what is given to what is useful for the readers.

I envision adding another section (in the future, when time allows) which shows how to run Promtail and the app outside of Docker containers. This would promote understanding of how to set up Promtail configuration.

So, in the future, I think there will be other sections that we might classify as extending the test environment to further evaluate Loki. I think that we might even use this setup (maybe replace flog with another app?) to be used as a LogQL query-writing practice area.


You can modify the flog app's log line generation by changing
its configuration.
Choose one of these two ways to apply a new configuration:

- To remove already-generated logs, restart the test environment with a new configuration.

1. Stop and clean up an existing test environment:
```
docker-compose down
```
1. Edit the `docker-compose.yaml` file. Within the YAML file, change the `flog.command` field's value to specify your flog output.
1. Instantiate the new test environment:
```
docker-compose up
```

- To keep already-generated logs in the running test environment, restart flog with a new configuration.

1. Edit the `docker-compose.yaml` file. Within the YAML file, change the `flog.command` field's value to specify your flog output.
1. Restart only the flog app within the currently-running test environment:
```
docker-compose up -d --force-recreate flog
```

78 changes: 0 additions & 78 deletions docs/sources/getting-started/get-logs-into-loki.md

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
57 changes: 0 additions & 57 deletions docs/sources/installation/simple-scalable-docker.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: Loki in Grafana
weight: 30
weight: 15
aliases:
- /docs/loki/latest/getting-started/grafana/
---
# Loki in Grafana

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: Troubleshooting
weight: 40
weight: 80
aliases:
- /docs/loki/latest/getting-started/troubleshooting/
---
# Troubleshooting Grafana Loki

Expand Down
7 changes: 7 additions & 0 deletions docs/sources/tools/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Tools
weight: 1050
---
# Tools

- [LogCLI](logcli/)
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: LogCLI
weight: 20
aliases:
- /docs/loki/latest/getting-started/logcli/
---
# LogCLI, Grafana Loki's command-line interface

Expand Down
12 changes: 12 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Loki examples

This directory contains examples and example configuration for Loki.

KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved

## Hosted logs in Grafana Cloud

You can use [Grafana Cloud](https://grafana.com/products/cloud/features/#cloud-logs) to avoid installing, maintaining, and scaling your own instance of Grafana Loki. The free forever plan includes 50GB of free logs. [Create an account to get started](https://grafana.com/auth/sign-up/create-user?pg=docs-loki&plcmt=in-text).

## Getting started with Loki

Configuration files in the `getting-started` directory are used by the [Loki getting started guide](https://grafana.com/docs/loki/latest/getting-started/).
4 changes: 4 additions & 0 deletions examples/getting-started/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Loki getting started guide configuration

This directory contains configuration to use with the [Loki getting started guide](https://grafana.com/docs/loki/latest/getting-started/)..
KMiller-Grafana marked this conversation as resolved.
Show resolved Hide resolved

Loading