Enable access_log rotate for ingress

[gyliu513]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see https://kubernetes.io/docs/tasks/debug-application-cluster/troubleshooting/.):

What keywords did you search in NGINX Ingress controller issues before filing this one? (If you have found any duplicates, you should instead reply there.):

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

/kind enhancement

In some stress cluster, we often found that ingress controller generate too many log messages and eat up the disk capacity quickly. It is because the access log is enabled.

After we set disable-access-log: true, the system come back and there are not too much logs for ingress controller, but the problem is that sometimes the end user need those logs for some debugging purpose, and disabling the access log make the end user has no chance to do such debugging.

One solution for this is that ingress controller should enable log rotate and log size limit, so that we can control the log in a more fine-grained mode.

There are actually some reference for this issue:

• Log rotate mechanism nginx/kubernetes-ingress#176
• https://github.com/kubernetes/kubernetes/pull/56979/files

@aledbf any comments for this?

/cc @todkap

NGINX Ingress controller version:

Kubernetes version (use kubectl version):

Environment:

• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

What happened:

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know:

[aledbf]

One solution for this is that ingress controller should enable log rotate and log size limit, so that we can control the log in a more fine-grained mode.

The problem with your request is the we don't log to file but to /dev/stdout and /dev/stderr so there's nothing to rotate. The only real solution for this scenario and don't suffer a loss in performance is to use the nginx syslog feature http://nginx.org/en/docs/syslog.html
This approach has two issues:

• kubectl logs will only return the output from the go code
• you are moving the log through the network to another place (you still need to handle it)

[luckyfengyong]

@gyliu513 I have the same concern as @aledbf. The log is dumped into /dev/stdout and /dev/stderr. We probably cannot control the size of the access log generated but only enable/disable it. For fine-grain control, even we are able to enable/disable for individual ingress resource, user is still able to enable the log and then generate garbage info to eat all the disk capacity of host.

The general log rotate of kuberentes in host level (https://kubernetes.io/docs/concepts/cluster-administration/logging/#logging-at-the-node-level) or in pod level (kubernetes/kubernetes#15478) should be the solution you expect.