Allow non-padded base64 data to be decoded by decode_base64_field (#2…

…7311) (#27554)

## What does this PR do?

This change allows the decoding of any raw base64 input strings that were previously encoded without standard padding character (`=`).

By stripping the padding, we can use `base64.RawStdEncoding.DecodeString` to decode the base64 string. This is easier than appending the padding characters.

Another attempt to fix this has been made in #25817, but that PR has been closed.

## Why is it important?

When attempting to decode the payload (middle) section of a JWT token, it was discovered that the decode was failing, because padding characters are not included in a JWT token string. Padding is not required in base64, so it makes sense to allow to decode both unpadded and padded strings.

Currently, there is a workaround for some beats by appending the `=`-signs in a javascript processor, but that isn't available in all beats and is an ugly workaround anyway. See https://medium.com/@guyromb/decode-jwt-traefik-access-logs-using-filebeat-95d935eb7c4f

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
(cherry picked from commit 9c4f7f9)

Co-authored-by: Michaël Arnauts <michael.arnauts@gmail.com>

CHANGELOG.next.asciidoc

@@ -309,6 +309,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Add cgroups V2 support {pull}27242[27242]
 - update ECS field definitions to ECS 1.11.0. {pull}27107[27107]
 - The disk queue is now GA. {pull}27515[27515]
+- Allow non-padded base64 data to be decoded by decode_base64_field {pull}27311[27311], {issue}27021[27021]
 
 *Auditbeat*
 

libbeat/processors/actions/decode_base64_field.go

@@ -20,6 +20,7 @@ package actions
 import (
 	"encoding/base64"
 	"fmt"
+	"strings"
 
 	"github.com/pkg/errors"
 
@@ -110,7 +111,7 @@ func (f *decodeBase64Field) decodeField(event *beat.Event) error {
 		return fmt.Errorf("invalid type for `from`, expecting a string received %T", value)
 	}
 
-	decodedData, err := base64.StdEncoding.DecodeString(base64String)
+	decodedData, err := base64.RawStdEncoding.DecodeString(strings.TrimRight(base64String, "="))
 	if err != nil {
 		return fmt.Errorf("error trying to decode %s: %v", base64String, err)
 	}

libbeat/processors/actions/decode_base64_field_test.go

@@ -87,6 +87,40 @@ func TestDecodeBase64Run(t *testing.T) {
 			},
 			error: false,
 		},
+		{
+			description: "simple field padded base64 decode from and to equals",
+			config: base64Config{
+				Field: fromTo{
+					From: "field1", To: "field1",
+				},
+				IgnoreMissing: false,
+				FailOnError:   true,
+			},
+			Input: common.MapStr{
+				"field1": "Y29ycmVjdCBwYWRkZWQgZGF0YQ==",
+			},
+			Output: common.MapStr{
+				"field1": "correct padded data",
+			},
+			error: false,
+		},
+		{
+			description: "simple field unpadded base64 decode from and to equals",
+			config: base64Config{
+				Field: fromTo{
+					From: "field1", To: "field1",
+				},
+				IgnoreMissing: false,
+				FailOnError:   true,
+			},
+			Input: common.MapStr{
+				"field1": "dW5wYWRkZWQgZGF0YQ",
+			},
+			Output: common.MapStr{
+				"field1": "unpadded data",
+			},
+			error: false,
+		},
 		{
 			description: "simple field bad data - fail on error",
 			config: base64Config{