Store the Fluentd logs to GCS bucket

[RadhaYarra193737]

What is a problem?

Hello,

I have requirement to store all the fluentbit logs to GCS bucket.
For this requirement i am doing the poc in one of my machine, I have followed this document
https://github.com/hartza-capital/fluent-bit-go-gcs

Result of the POC:

I am able to send the logs to GCS bucket , but i need to send as a compressed logs/Zip folder. To achieve what is the configuration change i need to make. I tried with "compress gzip " this field in my configuration but does not working.

Please check and let know your valuable feedback.

Configuration:

[SERVICE]
Flush 5
Daemon Off
Log_Level info
Parsers_File parsers.conf
Plugins_File plugins.conf
HTTP_Server Off
HTTP_Listen 0.0.0.0
HTTP_Port 2020

[INPUT]
Name cpu
Tag all-log-gcs
Interval_Sec 120
Buffer_Max_Size 1MB
#compress gzip

#[FILTER]

Name lua

Match *

Script /fluent-bit/scripts/zip_logs.lua

Call compress

[OUTPUT]
name stdout
match *
[OUTPUT]
Name gcs
Match *
Region asia-south1
Bucket fkh-sre-test
Prefix log
compress gzip
#Time_Format %Y-%m-%dT%H:%M:%S.%L%z

Describe the configuration of Fluentd

No response

Describe the logs of Fluentd

No response

Environment

- Fluentd version:
- TD Agent version:
- Fluent Package version:
- Docker image (tag):
- Operating system:
- Kernel version: