From 7a9804bce755607c208ff6033bc14c7f3e934ec3 Mon Sep 17 00:00:00 2001
From: Kaviraj <kavirajkanagaraj@gmail.com>
Date: Wed, 3 Mar 2021 14:10:41 +0100
Subject: [PATCH] doc(gcplog): Advanced log export filter example (#3421)

* doc(gcplog): Advanced log export filter example

* Update docs/sources/clients/promtail/gcplog-cloud.md

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>

Co-authored-by: Owen Diehl <ow.diehl@gmail.com>
---
 docs/sources/clients/promtail/gcplog-cloud.md | 39 ++++++++++++++++++-
 1 file changed, 37 insertions(+), 2 deletions(-)

diff --git a/docs/sources/clients/promtail/gcplog-cloud.md b/docs/sources/clients/promtail/gcplog-cloud.md
index 518bda3bcf72..3343ee2dd1d0 100644
--- a/docs/sources/clients/promtail/gcplog-cloud.md
+++ b/docs/sources/clients/promtail/gcplog-cloud.md
@@ -31,12 +31,12 @@ $ gcloud pubsub topics create cloud-logs
 We create a log sink to forward cloud logs into pubsub topic created before
 
 ```bash
-$ gcloud beta logging sinks create $SINK_NAME $SINK_LOCATION $OPTIONAL_FLAGS
+$ gcloud logging sinks create $SINK_NAME $SINK_LOCATION $OPTIONAL_FLAGS
 ```
 
 e.g:
 ```bash
-$ gcloud beta logging sinks create cloud-logs pubsub.googleapis.com/projects/my-project/topics/cloud-logs \
+$ gcloud logging sinks create cloud-logs pubsub.googleapis.com/projects/my-project/topics/cloud-logs \
 --log-filter='resource.type=("gcs_bucket")' \
 --description="Cloud logs"
 ```
@@ -44,6 +44,8 @@ $ gcloud beta logging sinks create cloud-logs pubsub.googleapis.com/projects/my-
 Above command also adds `log-filter` option which represents what type of logs should get into the destination `pubsub` topic.
 For more information on adding `log-filter` refer this [document](https://cloud.google.com/logging/docs/export/configure_export_v2#creating_sink)
 
+We cover more advanced `log-filter` [below](#Advanced-Log-filter)
+
 ## Create Pubsub subscription for Loki
 
 We create subscription for the pubsub topic we create above and `promtail` uses this subscription to consume log messages.
@@ -85,3 +87,36 @@ To delete all the old messages until now, set `--time` to current time.
 ```bash
 gcloud pubsub subscriptions seek projects/my-project/subscriptions/cloud-logs --time=$(date +%Y-%m-%dT%H:%M:%S)
 ```
+
+# Advanced log filter
+
+So far we've covered admitting GCS bucket logs into Loki, but often one may need to add multiple cloud resource logs and may also need to exclude unnecessary logs. The following is a more complex example.
+
+We use the `log-filter` option to include logs and the `exclusion` option to exclude them.
+
+## Use Case
+Include following cloud resource logs
+- GCS bucket
+- Kubernetes
+- IAM
+- HTTP Load balancer
+
+And we exclude specific HTTP load balancer logs based on payload and status code.
+
+```
+$ gcloud logging sinks create cloud-logs pubsub.googleapis.com/projects/my-project/topics/cloud-logs \
+--log-filter='resource.type=("gcs_bucket OR k8s_cluster OR service_account OR iam_role OR api OR audited_resource OR http_load_balancer")' \
+--description="Cloud logs" \
+--exclusion='name=http_load_balancer,filter=<<EOF
+resource.type="http_load_balancer"
+(
+	(
+		jsonPayload.statusDetails=("byte_range_caching" OR "websocket_closed")
+	)
+		OR
+	(
+		http_request.status=(101 OR 206)
+	)
+)
+EOF
+```