on the fly log analysis - parse the log only partially

[aivanise]

Hi,

I would like to run goaccess in a way that it will show me (on the console) what is going on in the log right now, i.e. from the moment i ran it, or shortly before I ran ir. If I give it a log file with -f then it parses it from the beginning of the log and only then goes into real time mode. I have tried piping to it or giving it a named pipe and then tailing the access log in the pipe, but it will never go into display mode, just parses the input, until I kill the tail and then remains static, i.e. never reopens a pipe.

An option like --since=timespec would be nice, meaning read the log file like usual but skip everything older than timespec.

thank you

[allinurl]

Thanks for suggesting this. Issue #582 and #428 should address the ability to partially parse a log.

As far as filtering from certain date, request #117 should deal with this.

Stay tuned for these changes!

[allinurl]

@aivanise I've pushed a few commits that give the ability to output live stats from a stream or an unclosed STDIN, i.e., tail -f as well as parsing multiple logs in live mode. For example:

goaccess --log-format=COMBINED vhost1.log vhost2.log

or

tail -f access.log | goaccess --log-format=COMBINED -

or even

tail -f access.log | goaccess --log-format=COMBINED -o report.html --real-time-html -

It also opens the possibility for live data filtering from the pipe such as:

tail -f access.log | grep -i --line-buffered 'firefox' | goaccess --log-format=COMBINED -

One thing to note is that tail -f will keep the pipe opened even when goaccess has already exited. For instance, tail -f syslog | grep -q 'cron' tail will exit on SIGPIPE, however it will only get SIGPIPE until an extra byte is written to the tail'd file. SIGTERM, SIGINT should close tail fine though.

Feel free to build from master to test this out, otherwise it will be pushed out in the upcoming release. Thanks.