How to edit config file for multiple log-formats

[ventouris]

The server I want to monitor provides me with quite a few different log files. All of them have also different format.

Those are the files I would like to monitor

Apache Access

ip_here - - [10/Oct/2019:03:49:17 +0000] "POST /wp-cron.php?doing_wp_cron=1570679356.8990719318389892578125 HTTP/1.0" 200 238 "https://mydomain.com/wp-cron.php?doing_wp_cron=1570679356.8990719318389892578125" "WordPress/5.2.3; https://mydomain.com"

Apache Error

[Thu Oct 10 03:49:23.604343 2019] [proxy_fcgi:error] [pid 19658] [client 127.0.0.1:35507] AH01071: Got error 'PHP message: Rank Math GSC API Error: POST https://www.googleapis.com/webmasters/v3/sites/https%3A%2F%2Fmydomain.com%2F/searchAnalytics/query 401 | {"error":{"errors":[{"domain":"global","reason":"authError","message":"Invalid Credentials","locationType":"header","location":"Authorization"}],"code":401,"message":"Invalid Credentials"}}\nPHP message: Rank Math GSC API Error: POST https://www.googleapis.com/webmasters/v3/sites/https%3A%2F%2Fmydomain.com%2F/searchAnalytics/query 401 | {"error":{"errors":[{"domain":"global","reason":"authError","message":"Invalid Credentials","locationType":"header","location":"Authorization"}],"code":401,"message":"Invalid Credentials"}}\nPHP message: Rank Math GSC API Error: POST https://www.googleapis.com/webmasters/v3/sites/https%3A%2F%2Fmydomain.com%2F/searchAnalytics/query 401 | {"error":{"errors":[{"domain":"global","reason":"authError","message":"Invalid Credentials","locationType":"header","location":"Authorization"}],"code":401,"message":"Invalid Credentials"}}\n', referer: https://mydomain.com/wp-admin/admin-ajax.php?action=wp_fetch_search_console_data&nonce=608b21aa5f

Nginx Status

status_code:200 ip_here [10/Oct/2019:03:49:17 +0000] POST /wp-cron.php?doing_wp_cron=1570679356.8990719318389892578125 HTTP/1.1

Nginx Access

ip_here - - [10/Oct/2019:02:48:32 +0000] "GET /wp-content/uploads/2019/06/media.png HTTP/1.1" 200 80005 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"

NGinx Error

2019/10/10 03:49:16 [error] 12904#12904: *515963 open() "/robots.txt" failed (2: No such file or directory), client: 157.55.39.18, server: mydomain.com, request: "GET /robots.txt HTTP/1.1", host: "mydomain.com"

PHP access log

ip_here - [08/Oct/2019:07:51:03 +0000] "GET /index.php" 200 0 - 971 30060 2.217 75497472 47.37% 9.02% "/robots.txt"

PHP app slow

[09-Oct-2019 04:22:44]  [pool zqxvgmwugn] pid 30216
script_filename = /wp-cron.php
[0x00007f5564021620] curl_exec() /wp-includes/Requests/Transport/cURL.php:162
[0x00007f55640211a0] request() /wp-includes/class-requests.php:379
[0x00007f5564020ec0] request() /wp-includes/class-http.php:384
[0x00007f5564020320] request() /wp-includes/class-http.php:613
[0x00007f5564020250] get() /wp-includes/http.php:170
[0x00007f55640201a0] wp_remote_get() /wp-content/plugins/wordfence/lib/wfScanEngine.php:2026
[0x00007f556401f940] startScan() /wp-content/plugins/wordfence/lib/wordfenceClass.php:313
[0x00007f556401eb60] dailyCron() /wp-includes/class-wp-hook.php:286
[0x00007f556401e7f0] apply_filters() /wp-includes/class-wp-hook.php:310
[0x00007f556401e740] do_action() /wp-includes/plugin.php:531
[0x00007f556401e550] do_action_ref_array() /wp-cron.php:133

As you can see, not only log-format is different, but also date and time format. I managed to setup the config file for Apache Access only, but then, the rest don't work (obviously). So, I was wondering if there is any way to have multiple formats, or if I can group some of them in the same command call.

[allinurl]

Currently it is not possible to parse different log formats. You can parse multiple logs with the same format though. #213 will address this issue.

[ventouris]

I see. If I judge from the date that the issue was created, it's a long shot that we will have this functionality soon :(

[allinurl]

Still there are a few major things that need to be implemented before this. Though, I do keep an eye on the most requested stuff and prioritize accordingly. Thanks for the heads up.

[allinurl]

Closing this as it is a dup. Feel free to reopen it if needed.