Backport of Fix audit-log encoding issue (CC-7337) into release/1.17.x

[hc-github-team-consul-core]

Backport

This PR is auto-generated from #20345 to be assessed for backporting due to the inclusion of the label backport/1.17.

🚨

Warning automatic cherry-pick of commits failed. If the first commit failed,
you will see a blank no-op commit below. If at least one commit succeeded, you
will see the cherry-picked commits up to, not including, the commit where
the merge conflict occurred.

The person who merged in the original PR is:
@absolutelightning
This person should manually cherry-pick the original PR into a new backport PR,
and close this one when the manual backport PR is merged in.

merge conflict error: unable to process merge commit: "d62a35f7e37d76e8db2d58e8e14d86fb6dce9e58", automatic backport requires rebase workflow

The below text is copied from the body of the original PR.

---

Description

Fixes bug - CC-7337, where we can see zip encoded characters in audit log files.

The audit-logs needs to be human-readable hence the changes in this PR is making sure that the audit log for any API call is only JSON encoded(not zip encoded). This is done by enforcing the request header Accept-Encoding: application/json only for audit logs. By doing so the zip handler will ignore encoding the response to zip.

Also found another bug, if we enforce the JSON in audit response, the audit-log handler is ignoring the 429 status-code hence converting it to 200.

Encoded log sample.

{"created_at":"2023-12-12T20:39:00.036046661Z","event_type":"audit","id":"consul-cluster-audit","organization_id":"11eaf78e-6956-5da6-b77e-0242ac11000d","payload":{"auth":{"accessor_id":"02dcb292-0cce-ab6e-4773-09dcc82cc932","description":"Managed Service Provider Token"},"id":"dbaa5048-a2ad-ce6a-9496-b3858047676a","request":{"endpoint":"/v1/operator/autopilot/health","host":"127.0.0.1:8500","operation":"GET","remote_addr":"127.0.0.1:57714","user_agent":"Go-http-client/1.1"},"response":{"error":"���k\u0014Q\f\u0007��%�ɐ����(b�x�҃���yy��ݕ�iQJ�wY�j��\u001eJ+x\f�\f�\u000f��\u0016�i�̟����f�\u001d������\u0017��Ny;*\f��J�\u001b��0|���70\u0000�\u0014�J��\u001c0�⑂Z�-�����\u0006\u001d��W\n\u0003�� \u0007Aq�\u001e9\u0011t���I�{\u0018����z�='\u001a��C{�S[�y�>L���F��K�������a�@\u0007皫N�\u001e������v��|����DNW{8�.t��A��ٶ�W\u0018�%���\u000e�t�\u001d\\������j�e","status":"429"},"stage":"OperationComplete","timestamp":"2023-12-12T20:39:00.03597823Z","type":"HTTPEvent","version":"1"},"project_id":"11eaf78e-69b6-5632-b77e-0242ac11000d"}
{"created_at":"2023-12-12T20:39:00.03617789Z","event_type":"audit","id":"consul-cluster-audit","organization_id":"11eaf78e-6956-5da6-b77e-0242ac11000d","payload":{"auth":{"accessor_id":"02dcb292-0cce-ab6e-4773-09dcc82cc932","description":"Managed Service Provider Token"},"id":"10af4ad8-0ca0-5dc5-2a01-eaa70353fa26","request":{"endpoint":"/v1/operator/autopilot/health","host":"127.0.0.1:8500","operation":"GET","remote_addr":"127.0.0.1:57714","user_agent":"Go-http-client/1.1"},"response":{"error":"���\u000f\u000e\u0010\u0012�,H�Bh0ap�\u0003�u�2�l�^\b�� 7�p\f���ƣK�V��d���\u0012�P��Y]�1\u0007=����H�[�W�\u0018,?1�\u0003�l[v�\rf�\u0011�U��2c��{�h\u0003�S1�'c\u0012�7&/�\u0011\u0007�\u0007\u001cչ*%�(��H6$T��lJ�&��KY��(�.r�^���8�a:F7:�L�ڪ��2b,��L�����ѩU!9�+$O�8.�_5^�t\u0014����`T*X�(\n1���&�Ɏ�b<\u0012ʲ\u0007�^���\u0018\u000f�>����\u0017G����\u0000","status":"429"},"stage":"OperationComplete","timestamp":"2023-12-12T20:39:00.036140264Z","type":"HTTPEvent","version":"1"},"project_id":"11eaf78e-69b6-5632-b77e-0242ac11000d"}

Testing & Reproduction steps

Manual steps on local -

• In ENT repo - Replace line with

		health.Servers = append(health.Servers, srvHealth)
		health.Servers = append(health.Servers, srvHealth)
		health.Servers = append(health.Servers, srvHealth)
		health.Servers = append(health.Servers, srvHealth)

• Run the ENT binary with licence and enable audit log using config -

audit = {
	enabled =  true
	sink = {
	 file-sink = {
	    name = "file-sink"
	    type = "file"
	    format = "json"
	    path = "/tmp/audit.json"
	    delivery_guarantee = "best-effort"
	    mode = "640"
	    rotate_bytes = 67108864
	    rotate_duration = "1h",
	    rotate_max_files = 176
	 }
	}
}
acl = {
  enabled = true
  default_policy = "deny"
  enable_token_persistence = true
} 

• Run consul acl bootstrap
• Take the token and replace it in the following code snippet for the value of X-Consul-Token -

package main

import (
	"fmt"
	"net/http"
)

func makeReq() {
	req, _ := http.NewRequest("GET", "http://localhost:8500/v1/operator/autopilot/health", nil)
	req.Header.Add("X-Consul-Token", "token")
	client := &http.Client{}
	res, _ := client.Do(req)
	fmt.Println(res)
}

func main() {
	makeReq()
}

• Postman could also be used instead of above code
• Check the audit logs file generated. It will be generated like below -

{"created_at":"2024-01-25T13:08:34.896035+05:30","event_type":"audit","payload":{"id":"e36c7030-96bd-c16f-df51-ef75ecba4109","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:08:34.895913+05:30","auth":{"accessor_id":"bbca76a4-fad0-e6d1-4fd2-e4bacc416a50","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:08:17.326937+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50249","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"stage":"OperationStart"}}
{"created_at":"2024-01-25T13:08:34.899233+05:30","event_type":"audit","payload":{"id":"d1e3404c-88c2-6056-e763-72edf6dbe6c3","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:08:34.8991+05:30","auth":{"accessor_id":"bbca76a4-fad0-e6d1-4fd2-e4bacc416a50","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:08:17.326937+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50249","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"response":{"status":"429","error":"\ufffd\ufffd\ufffdk\ufffd0\u0014\ufffd\ufffd\ufffd\u0015\ufffd\ufffdMIckjnccS\u0018\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\u003c\ufffdP[H\ufffd2\u0011\ufffd\ufffd\ufffd:\u001c\ufffd\ufffd\ufffd[s\n\ufffd\ufffd\ufffd=\ufffd\u001c'\ufffd1\u0006K\ufffd\ufffd\ufffd\u000e\ufffd\ufffd\u0016KG\ufffd\ufffd\ufffd\u0011\ufffd\ufffd\ufffd\ufffd\ufffd%Y\ufffdr\u0002\ufffdķ\ufffd\ufffdm\ufffd:\ufffd\ufffd\ufffd\u001ft\ufffdx\ufffd\ufffd\ufffd\ufffd\u0003h\u0006\ufffd\ufffd\ufffd\ufffd69Ϸ\u000b\ufffd\u0013R\ufffd\ufffdyb\ufffd\"\ufffd\ufffdH\ufffdd,M\u0002\ufffd\ufffd\ufffd\u0017\ufffdw\ufffd\u0000ݎܮ-̡\ufffd|)\ufffd^\ufffd|\ufffd^dC\ufffd\ufffd1\ufffd\\w\u000b\ufffdR\u0005\"\u0010A\ufffd\ufffd\ufffd\u0010CcJv\ufffdz\ufffdM\ufffdŲhihY\ufffduE]\ufffd]A\ufffd\u0004W\u001dτ\ufffd,h\ufffdmCC\r\ufffd\ufffd\ufffd+\ufffd\ufffd\ufffd\n\ufffd\ufffd\n\ufffd\ufffd\u0019\ufffd=h\u0026oH\ufffd\ufffd\ufffdg\ufffdE\u0003\ufffd\u0007\ufffd"},"stage":"OperationComplete"}}
{"created_at":"2024-01-25T13:08:34.899337+05:30","event_type":"audit","payload":{"id":"4b32ca51-cab2-8687-ab9f-c8c63eb06cbf","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:08:34.899118+05:30","auth":{"accessor_id":"bbca76a4-fad0-e6d1-4fd2-e4bacc416a50","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:08:17.326937+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50249","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"response":{"status":"429","error":"\ufffd\ufffd\ufffd\ufffd\ufffduP\ufffdqSRZ\ufffdف\u00142\ufffd\"\ufffd2΄\ufffd3\ufffdc\ufffd\u0006\ufffd\ufffdi:\ufffd\u001cI\ufffd$G\ufffd#\ufffd� \ufffd\ufffd\u003e\u0026\ufffd\ufffd\u0000"},"stage":"OperationComplete"}}
{"created_at":"2024-01-25T13:08:34.899417+05:30","event_type":"audit","payload":{"id":"1774db2a-62a5-66f9-b795-11bb57674e92","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:08:34.899135+05:30","auth":{"accessor_id":"bbca76a4-fad0-e6d1-4fd2-e4bacc416a50","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:08:17.326937+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50249","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"response":{"status":"429","error":"\u0000\u0000\ufffd\ufffd"},"stage":"OperationComplete"}}
{"created_at":"2024-01-25T13:08:34.900259+05:30","event_type":"audit","payload":{"id":"384ec7cc-2952-0994-9b7c-69ca055e64da","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:08:34.899144+05:30","auth":{"accessor_id":"bbca76a4-fad0-e6d1-4fd2-e4bacc416a50","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:08:17.326937+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50249","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"response":{"status":"429","error":"\ufffd$\u001b\ufffdr\u0007\u0000\u0000"},"stage":"OperationComplete"}}
{"created_at":"2024-01-25T13:08:34.900446+05:30","event_type":"audit","payload":{"id":"01eb3bfa-d5c5-ed4a-2a1f-e427480a105b","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:08:34.896211+05:30","auth":{"accessor_id":"bbca76a4-fad0-e6d1-4fd2-e4bacc416a50","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:08:17.326937+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50249","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"response":{"status":"429","error":"\u001f\ufffd\u0008\u0000\u0000\u0000\u0000\u0000\u0000\ufffd"},"stage":"OperationComplete"}}

After this fix the log file will be generated without Non Ascii Characters -

{"created_at":"2024-01-25T13:09:51.131972+05:30","event_type":"audit","payload":{"id":"7fe20502-c6c8-5584-7c83-eeed8e73b07b","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:09:51.130662+05:30","auth":{"accessor_id":"a1bfe219-6d07-bc21-ea62-e4bb8c1ccafe","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:09:37.657552+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50269","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"stage":"OperationStart"}}
{"created_at":"2024-01-25T13:09:51.132719+05:30","event_type":"audit","payload":{"id":"6eef6765-06f8-f638-7e6b-1a36307f52a0","version":"1","type":"HTTPEvent","timestamp":"2024-01-25T13:09:51.132605+05:30","auth":{"accessor_id":"a1bfe219-6d07-bc21-ea62-e4bb8c1ccafe","description":"Bootstrap Token (Global Management)","create_time":"2024-01-25T13:09:37.657552+05:30"},"request":{"operation":"GET","endpoint":"/v1/operator/autopilot/health","remote_addr":"127.0.0.1:50269","user_agent":"Go-http-client/1.1","host":"localhost:8500"},"response":{"status":"429","error":"{\n    \"Healthy\": false,\n    \"FailureTolerance\": 0,\n    \"Servers\": [\n        {\n            \"ID\": \"5756debc-cf90-8e72-1c8d-9548423252d8\",\n            \"Name\": \"asheshvidyut-H2GX766V9T\",\n            \"Address\": \"127.0.0.1:8300\",\n            \"SerfStatus\": \"alive\",\n            \"Version\": \"1.18.0\",\n            \"Leader\": true,\n            \"LastContact\": \"0s\",\n            \"LastTerm\": 2,\n            \"LastIndex\": 24,\n            \"Healthy\": true,\n            \"Voter\": true,\n            \"StableSince\": \"2024-01-25T07:39:36Z\"\n        },\n        {\n            \"ID\": \"5756debc-cf90-8e72-1c8d-9548423252d8\",\n            \"Name\": \"asheshvidyut-H2GX766V9T\",\n            \"Address\": \"127.0.0.1:8300\",\n            \"SerfStatus\": \"alive\",\n            \"Version\": \"1.18.0\",\n            \"Leader\": true,\n            \"LastContact\": \"0s\",\n            \"LastTerm\": 2,\n            \"LastIndex\": 24,\n            \"Healthy\": true,\n            \"Voter\": true,\n            \"StableSince\": \"2024-01-25T07:39:36Z\"\n        },\n        {\n            \"ID\": \"5756debc-cf90-8e72-1c8d-9548423252d8\",\n            \"Name\": \"asheshvidyut-H2GX766V9T\",\n            \"Address\": \"127.0.0.1:8300\",\n            \"SerfStatus\": \"alive\",\n            \"Version\": \"1.18.0\",\n            \"Leader\": true,\n            \"LastContact\": \"0s\",\n            \"LastTerm\": 2,\n            \"LastIndex\": 24,\n            \"Healthy\": true,\n            \"Voter\": true,\n            \"StableSince\": \"2024-01-25T07:39:36Z\"\n        },\n        {\n            \"ID\": \"5756debc-cf90-8e72-1c8d-9548423252d8\",\n            \"Name\": \"asheshvidyut-H2GX766V9T\",\n            \"Address\": \"127.0.0.1:8300\",\n            \"SerfStatus\": \"alive\",\n            \"Version\": \"1.18.0\",\n            \"Leader\": true,\n            \"LastContact\": \"0s\",\n            \"LastTerm\": 2,\n            \"LastIndex\": 24,\n            \"Healthy\": true,\n            \"Voter\": true,\n            \"StableSince\": \"2024-01-25T07:39:36Z\"\n        }\n    ]\n}"},"stage":"OperationComplete"}}

PR Checklist

• updated test coverage
• external facing docs updated
• appropriate backport labels added
• not a security concern

---

Overview of commits

• d62a35f - d883550 - 87b6971 - c418689 - 9008b22 - 07af66c - 6ff9fe9 - d98d6af - 5b1cc3c - a64a031 - 9377ca6 - e3f2751 - 01ac423 - 8fd5619 - 54f5c2d - 0f247ba - e6dbfb8 - c12c127 - aace885 - 35f4f20 - 2b9199d - e0062ec - d652e21 - fc7bfda - da4b4fb - 0f99aa6 - 28f6aa6 - 029632a - 7e41089 - 13d6350

[hashicorp-cla]

All committers have signed the CLA.

[github-team-consul-core-pr-approver]

Auto approved Consul Bot automated PR