Docs: revise getting started guide (#5939)

* Revise the getting started section

* Docs: revise getting started guide

* Intermediate commit of getting started material.

* Intermediate revision of the getting started guide.

    Add instructions to download, configure, and deploy the test environment.

* Add sample queries and instructions for breaking down the test env

* Revise prose using reviewer comments.

* update to use promtail and flog in the docker compose file

Signed-off-by: Edward Welch <edward.welch@grafana.com>

* Revise guide to put all parts into Docker containers.

* Add README files for examples directory. Revise getting started instructions.

* Address DylanGuedes review comments

* Getting started guide: incorporate suggestions from ssncferreira

Co-authored-by: Edward Welch <edward.welch@grafana.com>
(cherry picked from commit c4ebfd3)

docs/sources/getting-started/_index.md

@@ -1,14 +1,142 @@
 ---
 title: Getting started
 weight: 300
+description: "This guide assists the reader to create and use a simple Loki cluster for testing and evaluation purposes."
+aliases:
+    - /docs/loki/latest/getting-started/get-logs-into-loki/
 ---
+
 # Getting started with Grafana Loki
 
 > **Note:** You can use [Grafana Cloud](https://grafana.com/products/cloud/features/#cloud-logs) to avoid installing, maintaining, and scaling your own instance of Grafana Loki. The free forever plan includes 50GB of free logs. [Create an account to get started](https://grafana.com/auth/sign-up/create-user?pg=docs-loki&plcmt=in-text).
 
-1. [Getting Logs Into Loki](get-logs-into-loki/)
-1. [Grafana](grafana/)
-1. [LogCLI](logcli/)
-1. [Labels](labels/)
-1. [Troubleshooting](troubleshooting/)
+This guide assists the reader to create and use a simple Loki cluster.
+The cluster is intended for testing, development, and evaluation;
+it will not meet most production requirements.
+
+The test environment runs the [flog](https://github.com/mingrammer/flog) app to generate log lines.
+Promtail is the test environment's agent (or client) that captures the log lines and pushes them to the Loki cluster through a gateway.
+In a typical environment, the log-generating app and the agent run together, but in locations distinct from the Loki cluster. This guide runs each piece of the test environment locally, in Docker containers.
+
+Grafana provides a way to pose queries against the logs stored in Loki and visualize query results.
+
+![Simple scalable deployment test environment](simple-scalable-test-environment.png)
+
+The test environment uses Docker compose to instantiate these parts, each in its own container: 
+
+- One [single scalable deployment](../fundamentals/architecture/deployment-modes/) mode **Loki** instance has:
+    - One Loki read component
+    - One Loki write component
+    - **Minio** is Loki's storage back end in the test environment.
+- The **gateway** receives requests and redirects them to the appropriate container based on the request's URL.
+- **Flog** generates log lines.
+- **Promtail** scrapes the log lines from flog, and pushes them to Loki through the gateway.
+- **Grafana** provides visualization of the log lines captured within Loki.
+
+## Prerequisites
+
+- [Docker](https://docs.docker.com/install)
+- [Docker Compose](https://docs.docker.com/compose/install)
+
+## Obtain the test environment
+
+1. Create a directory called `evaluate-loki` for the test environment. Make `evaluate-loki` your current working directory:
+    ```bash
+    mkdir evaluate-loki
+    cd evaluate-loki
+    ```
+1. Download `loki-config.yaml`, `promtail-local-config.yaml`, and `docker-compose.yaml`:
+
+    ```bash
+    wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
+    wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/promtail-local-config.yaml -O promtail-local-config.yaml
+    wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml
+    ```
+
+## Deploy the test environment
+
+All shell commands are issued from the `evaluate-loki` directory.
+
+1. With `evaluate-loki` as the current working directory, deploy the test environment using `docker-compose`:
+    ```bash
+    docker-compose up -d
+    ```
+1. (Optional) Verify that the Loki cluster is up and running. The read component returns `ready` when you point a web browser at http://localhost:3101/ready. The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready.
+The write component returns `ready` when you point a web browser at http://localhost:3102/ready. The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready.
+
+## Use Grafana and the test environment
+
+Use [Grafana](https://grafana.com/docs/grafana/latest/) to query and observe the log lines captured in the Loki cluster by navigating a browser to http://localhost:3000.
+The Grafana instance has Loki configured as a [datasource](https://grafana.com/docs/grafana/latest/datasources/loki/).
+
+Click on the Grafana instance's [Explore](https://grafana.com/docs/grafana/latest/explore/) icon to bring up the explore pane.
+
+Use the Explore dropdown menu to choose the Loki datasource and bring up the Loki query browser.
+
+Try some queries.
+Enter your query into the **Log browser** box, and click on the blue **Run query** button.
+
+To see all the log lines that flog has generated:
+```
+{container="evaluate-loki_flog_1"}
+```
+
+The flog app will generate log lines for invented HTTP requests.
+To see all `GET` log lines, enter the query:
+
+```
+{container="evaluate-loki_flog_1"} |= "GET"
+```
+For `POST` methods:
+```
+{container="evaluate-loki_flog_1"} |= "POST"
+```
+
+To see every log line with a 401 status (unauthorized error): 
+```
+{container="evaluate-loki_flog_1"} | json | status="401"
+```
+To see every log line other than those that contain the value 401: 
+```
+{container="evaluate-loki_flog_1"} != "401"
+```
+
+Refer to [query examples](../logql/query_examples/) for more examples.
+
+## Stop and clean up the test environment
+
+To break down the test environment:
+
+- Close the Grafana browser window
+
+- Stop and remove all the Docker containers. With `evaluate-loki` as the current working directory:
+    ```bash
+    docker-compose down
+    ```
+
+## Modifying the flog app output
+
+You can modify the flog app's log line generation by changing
+its configuration.
+Choose one of these two ways to apply a new configuration:
+
+- To remove already-generated logs, restart the test environment with a new configuration.
+
+    1. Stop and clean up an existing test environment:
+        ```
+        docker-compose down
+        ```
+    1. Edit the `docker-compose.yaml` file.  Within the YAML file, change the `flog.command` field's value to specify your flog output.
+    1. Instantiate the new test environment:
+        ```
+        docker-compose up
+        ```
+
+- To keep already-generated logs in the running test environment, restart flog with a new configuration.
+
+    1. Edit the `docker-compose.yaml` file.  Within the YAML file, change the `flog.command` field's value to specify your flog output.
+    1. Restart only the flog app within the currently-running test environment:
+        ```
+        docker-compose up -d --force-recreate flog
+        ```
 

docs/sources/getting-started/grafana.md → docs/sources/operations/grafana.md

@@ -1,6 +1,8 @@
 ---
 title: Loki in Grafana
-weight: 30
+weight: 15
+aliases:
+    - /docs/loki/latest/getting-started/grafana/
 ---
 # Loki in Grafana
 

docs/sources/getting-started/troubleshooting.md → docs/sources/operations/troubleshooting.md

@@ -1,6 +1,8 @@
 ---
 title: Troubleshooting
-weight: 40
+weight: 80
+aliases:
+    - /docs/loki/latest/getting-started/troubleshooting/
 ---
 # Troubleshooting Grafana Loki
 

docs/sources/tools/_index.md

@@ -0,0 +1,7 @@
+---
+title: Tools
+weight: 1050
+---
+# Tools
+
+- [LogCLI](logcli/)

docs/sources/getting-started/logcli.md → docs/sources/tools/logcli.md

@@ -1,6 +1,8 @@
 ---
 title: LogCLI
 weight: 20
+aliases:
+    - /docs/loki/latest/getting-started/logcli/
 ---
 # LogCLI, Grafana Loki's command-line interface
 

examples/README.md

@@ -0,0 +1,11 @@
+# Loki examples
+
+This directory contains examples and example configuration for Loki.
+
+## Hosted logs in Grafana Cloud
+
+You can use [Grafana Cloud](https://grafana.com/products/cloud/features/#cloud-logs) to avoid installing, maintaining, and scaling your own instance of Grafana Loki. The free forever plan includes 50GB of free logs. [Create an account to get started](https://grafana.com/auth/sign-up/create-user?pg=docs-loki&plcmt=in-text).
+
+## Getting started with Loki
+
+Configuration files in the `getting-started` directory are used by the [Loki getting started guide](https://grafana.com/docs/loki/latest/getting-started/).

examples/getting-started/README.md

@@ -0,0 +1,4 @@
+# Loki getting started guide configuration
+
+This directory contains configuration to use with the [Loki getting started guide](https://grafana.com/docs/loki/latest/getting-started/).
+

production/simple-scalable/docker-compose.yaml → examples/getting-started/docker-compose.yaml

@@ -37,15 +37,9 @@ services:
 
   promtail:
     image: grafana/promtail:2.5.0
-    logging: &logging
-      driver: loki-compose
-      options:
-        loki-url: "http://localhost:3100/loki/api/v1/push"
-        loki-retries: "1"
-        loki-tenant-id: "tenant1"
     volumes:
-      - /var/log:/var/log
-      - ./promtail-config.yaml:/etc/promtail/config.yaml:ro
+      - ./promtail-local-config.yaml:/etc/promtail/config.yaml:ro
+      - /var/run/docker.sock:/var/run/docker.sock
     command: -config.file=/etc/promtail/config.yaml
     depends_on:
       - gateway
@@ -66,8 +60,6 @@ services:
       - MINIO_SECRET_KEY=supersecret
       - MINIO_PROMETHEUS_AUTH_TYPE=public
       - MINIO_UPDATE=off
-    logging:
-      <<: *logging
     ports:
       - 9000
     volumes:
@@ -77,8 +69,6 @@ services:
 
   grafana:
     image: grafana/grafana:latest
-    logging:
-      <<: *logging
     environment:
       - GF_PATHS_PROVISIONING=/etc/grafana/provisioning
       - GF_AUTH_ANONYMOUS_ENABLED=true
@@ -109,8 +99,6 @@ services:
       - loki
 
   gateway:
-    logging:
-      <<: *logging
     image: nginx:latest
     depends_on:
       - read
@@ -173,3 +161,9 @@ services:
       - "3100:3100"
     networks:
       - loki
+
+  flog:
+    image: mingrammer/flog
+    command: -f json -d 1s -l
+    networks:
+      - loki