Safe defaults for agent file logging

[schmichael]

I missed a couple things when reviewing #6429:

1. It would save users some calculator usage to allow them to specify the max file size in MB instead of bytes. That would also match our jobspec: https://www.nomadproject.io/docs/job-specification/logs.html#max_file_size
2. Defaulting to an unbounded file size (0) seems unsafe. Let's default to 10 files of 10 MB. That way it keeps the storage overhead of Nomad more predictable and easier to factor into the client.reserved stanza.

As a stretch goal we could probably also now implement this as an alternate Sink on hclogger once #6499 lands. That would allow logging at a different log level as well so you could log to stdout (and therefore to your central aggregator) at INFO, but have 100MB worth of DEBUG logs locally in case something went wrong. Probably not worth worrying about in the 0.10.x timeline. Maybe we'll even get this for free once Consul implements hclogger.

[endocrimes]

Currently we have the same defaults as Consul, which potentially has some value for someone running both (most nomad users I would assume).

We might want to work with consul to improve this across both.

[schmichael]

Consul does not appear to support configuring file logging rotation from the config file -- or at least it's not documented that I see: https://www.consul.io/docs/agent/options.html#log_file

Update: That was just a docs mistake that they're fixing: hashicorp/consul#6705

So by only supporting via the config file we're already diverging from Consul. I think safe defaults are worth diverging in behavior -- especially when it makes us more internally consistent with our jobspec logging behavior.

[schmichael]

Going to close since we're shipping 0.10.2 with it the way it is. We can reopen based on user demand/experience.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.