logdog: Filter sensitive settings from API output

[zmrow]

Description of changes:

    Previously, `logdog` used the `exec` mode to run `apiclient` and dump
    everything from the datastore.  Since a few settings now exist that can
    store sensitive data, we need to filter them from being written to disk.
    
    In order to make this change, `logdog` needed to be trained to
    understand the data it was gathering, in order to filter it.  A new
    `settings` mode was added specifically for gathering settings.  This
    mode uses the `apiclient` and `datastore` libraries to be able to
    de/serialize the output, filter it, and then serialize it back to
    disk

For now, we're filtering all user-data fields, the Kubernetes bootstrap token, and https proxy.

Testing done:

• Ran a vmware-k8s-1.20 with user data for both host containers, a bootstrap container with user data, and the required k8s token. Ran logdog and inspected the output, which did not contain any user data, or k8s token. Relevant sections below:

...
  "kubernetes": {
    "cluster-certificate": <removed for clarity's sake, it's really long :) >,
    "api-server": "https://198.18.5.50:6443",
    "authentication-mode": "tls",
    "standalone-mode": false,
    "server-tls-bootstrap": false,
    "cloud-provider": "external",
    "cluster-dns-ip": "10.96.0.10",
    "cluster-domain": "cluster.local",
    "node-ip": "198.19.128.120",
    "pod-infra-container-image": "k8s.gcr.io/pause:3.2"
  }
  "host-containers": {
    "control": {
      "source": "public.ecr.aws/bottlerocket/bottlerocket-control:v0.5.2",
      "enabled": true,
      "superpowered": false
    },
    "admin": {
      "source": "public.ecr.aws/bottlerocket/bottlerocket-admin:v0.7.2",
      "enabled": true,
      "superpowered": true
  "bootstrap-containers": {
    "bear": {
      "source": "<my private container>",
      "mode": "off"
    }
  },
...

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[zmrow]

^ Fixed up the tests now that things are async.

[webern]

It might not be obvious to a user that these settings were removed, which could lead someone astray if the issue under investigation is related to one of the sensitive settings. I'm not sure what to suggest though, this is the lesser of evils for sure.

[zmrow]

^ Implemented @tjkirch 's suggestions

[zmrow]

It might not be obvious to a user that these settings were removed, which could lead someone astray if the issue under investigation is related to one of the sensitive settings. I'm not sure what to suggest though, this is the lesser of evils for sure.

Good point, but I would agree this is the lesser of evils. A user should still be able to fetch these settings via apiclient if they're needed for troubleshooting.

[arnaldo2792]

Should the cluster certificate be added to this list?

[zmrow]

I don't think so - I believe it's public.