feat: clickhouse logger

[zhendongcmss]

What this PR does / why we need it:

feat: push access log to clickhouse DB

Pre-submission checklist:

• Did you explain what problem does this PR solve? Or what new features have been added?
• Have you added corresponding test cases?
• Have you modified the corresponding document?
• Is this PR backward compatible? If it is not backward compatible, please discuss on the mailing list first

[zhendongcmss]

Hi, @spacewander
How fix this test case ?
https://github.com/apache/apisix/runs/4989817705?check_suite_focus=true#step:7:22

Installing collected packages: zhon
Successfully installed zhon-1.1.5
find broken newline in file: docs/zh/latest/plugins/clickhouse-logger.md
cat: /tmp/error.log: No such file or directory
Error: Process completed with exit code 1.

[spacewander]

Hi, @spacewander How fix this test case ? apache/apisix/runs/4989817705?check_suite_focus=true#step:7:22

Installing collected packages: zhon
Successfully installed zhon-1.1.5
find broken newline in file: docs/zh/latest/plugins/clickhouse-logger.md
cat: /tmp/error.log: No such file or directory
Error: Process completed with exit code 1.

Sorry, it is the bug in our lint script. Fixed in #6239

[tzssangglass]

hi @zhendongcmss CI is broken

[zhendongcmss]

hi @zhendongcmss CI is broken

@spacewander @tzssangglass
sorry， I don't know how to fix. I need help, can you tell me how to fix it ?

#   Failed test 'TEST 4: add plugin on routes - status code ok'
#   at /apisix/test-nginx/lib/Test/Nginx/Socket.pm line 948.
#          got: '401'
#     expected: '200'

#   Failed test 'TEST 5: access local server - status code ok'
#   at /apisix/test-nginx/lib/Test/Nginx/Socket.pm line 948.
#          got: '404'
#     expected: '200'

#   Failed test 'TEST 5: access local server - response_body - response is expected (repeated req 0, req 0)'
#   at /apisix/test-nginx/lib/Test/Nginx/Socket.pm line 1589.
#          got: '{"error_msg":"404 Route Not Found"}
# '
#     expected: 'opentracing

[tzssangglass]

hi @zhendongcmss CI is broken

@spacewander @tzssangglass sorry， I don't know how to fix. I need help, can you tell me how to fix it ?

#   Failed test 'TEST 4: add plugin on routes - status code ok'
#   at /apisix/test-nginx/lib/Test/Nginx/Socket.pm line 948.
#          got: '401'
#     expected: '200'

#   Failed test 'TEST 5: access local server - status code ok'
#   at /apisix/test-nginx/lib/Test/Nginx/Socket.pm line 948.
#          got: '404'
#     expected: '200'

#   Failed test 'TEST 5: access local server - response_body - response is expected (repeated req 0, req 0)'
#   at /apisix/test-nginx/lib/Test/Nginx/Socket.pm line 1589.
#          got: '{"error_msg":"404 Route Not Found"}
# '
#     expected: 'opentracing

My comments above solved the 401 problem with test 4, you can continue to see the problem with test 5, which is related to the code and configuration.

[spacewander]

I have reconsidered the test.
What about using error_log like

apisix/t/plugin/loggly.t

Lines 572 to 574 in 537a8da

	--- error_log
	loggly body: {"route_id":"1"}
	loggly tags: "apisix"

?

[zhendongcmss]

It doesn't work.

[zhendongcmss]

It doesn't work. I don't know why. I think --- grep_error_log_out is easier to use.

[zhendongcmss]

Use --- error_log fialed. https://github.com/apache/apisix/runs/5197064703?check_suite_focus=true#step:14:891

[spacewander]

Suggested change

	"batch_max_size":1000,
	"batch_max_size":1,

So the report can be triggered in the CI

[zhendongcmss]

I thought there was a timer that called this callback function periodically and also if the batch_max_size value was reached. But I added a wait of --- wait: 5 below, it seem doesn'twork.

[spacewander]

Could you apply this patch:

--- t/plugin/clickhouse-logger.t
+++ t/plugin/clickhouse-logger.t
@@ -149,7 +149,9 @@ property "endpoint_addr" is required
                                 "password": "a",
                                 "database": "default",
                                 "logtable": "t",
-                                "endpoint_addr": "http://127.0.0.1:10420/clickhouse-logger/test"
+                                "endpoint_addr": "http://127.0.0.1:10420/clickhouse-logger/test",
+                                "batch_max_size":1,
+                                "inactive_timeout":1
                             }
                         },
                         "upstream": {
@@ -173,7 +175,7 @@ property "endpoint_addr" is required
                             },
                             "plugins":{
                                 "clickhouse-logger":{
-                                    "batch_max_size":1000,
+                                    "batch_max_size":1,
                                     "max_retry_count":0,
                                     "retry_delay":1,
                                     "ssl_verify":true,
@@ -185,7 +187,7 @@ property "endpoint_addr" is required
                                     "name":"clickhouse-logger",
                                     "database":"default",
                                     "logtable":"t",
-                                    "inactive_timeout":5
+                                    "inactive_timeout":1
                                 }
                             },
                             "id":"1"
@@ -213,7 +215,7 @@ GET /opentracing
 opentracing
 --- error_log
 clickhouse body: INSERT INTO t FORMAT JSONEachRow
-clickhouse headers: X-ClickHouse-Key:a
-clickhouse headers: X-ClickHouse-User:default
-clickhouse headers: X-ClickHouse-Database:default
---- wait: 5
+clickhouse headers: x-clickhouse-key:a
+clickhouse headers: x-clickhouse-user:default
+clickhouse headers: x-clickhouse-database:default
+--- wait: 2

Now the test should pass.
There is another issue, the headers we get are all lowercase.

[spacewander]

@zhendongcmss
Let's add the logger in https://github.com/apache/apisix/blob/master/README.md!

[juzhiyuan]

Hi @zhendongcmss, here have some Chinese 😄 Would you like to translate them?

[juzhiyuan]

Hi, @yzeng25, do you have interest to do this? 🤗

[membphis]

I created an issue, let's fix it: #6414