Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an alternative path for user.toml #3814

Merged
merged 5 commits into from
Nov 6, 2017
Merged

Add an alternative path for user.toml #3814

merged 5 commits into from
Nov 6, 2017

Commits on Nov 3, 2017

  1. Refactor Cfg::new to not use Cfg::default() 

    In some follow up commit I'd like to add another field, that does not
have an obvious default value. Either I could work it around by
putting it inside Option (and then unwrap() the field with assumption
that after Cfg::new() finishes, it will never be None) or just stop
using Cfg::default().

I went with the latter.

Signed-off-by: Krzesimir Nowak <krzesimir@kinvolk.io>
    @krnowak
    krnowak committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    4b89232 View commit details
    Browse the repository at this point in the history

  2. Add an alternative path for user.toml 

    This commit makes the supervisor to look for user configuration in two
locations: /hab/svc/<pkgname>/user.toml and
/hab/user/<pkgname>/config/user.toml. If the first path is missing, it
will try to load the configuration from the second one.

The reason is to work around an issue in Kubernetes/Docker/Linux
kernel.

In the Habitat operator we use the Kubernetes Secret feature for
putting user configuration as an initial configuration and as a source
of configuration updates. Initially it worked fine, until we
discovered that Kubernetes actually mounts a directory at
/hab/svc/<pkgname> to put user.toml there, so all the previous
contents there were effectively hidden.

We worked it around with a SubPath feature of Kubernetes, which
basically instead of mounting a directory at /hab/svc/<pkgname>,
bind-mounts some file on the host to the /hab/svc/<pkgname>/user.toml
path inside the Kubernetes pod.

This unfortunately breaks updating the configuration via the
Kubernetes Secret. This is because Kubernetes tries hard to make the
change of the Secret atomic. The atomicity of the change is achieved
with renaming. Doing the rename changes the inode of the target file
and since the bind mounts are based on inodes, the change is not
reflected in the pod.

It may take some time before we come up with a solution for this
problem and the solution to be released, so for now we would like to
add the second path for the user configuration. The new
/hab/user/<pkgname>/config directory can only contain user.toml file,
so it is safe to mount a directory here with our own copy of the file,
and we can stop using SubPath feature, so the updates should work
again.

Signed-off-by: Krzesimir Nowak <krzesimir@kinvolk.io>
    @krnowak
    krnowak committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    a20498a View commit details
    Browse the repository at this point in the history

  3. Make service config code easier to test 

    Introduce the PackageConfigPaths trait, which will allow us to
override the paths where the user configuration should be put for
testing purposes. Before that, the paths were basically hardcoded to
something under `/hab`. There is an environment variable
(`TESTING_FS_ROOT`) for changing the filesystem root, where `hab`
directory is being placed, but it is not really usable from unit
tests.

Signed-off-by: Krzesimir Nowak <krzesimir@kinvolk.io>
    @krnowak
    krnowak committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    5ff6caf View commit details
    Browse the repository at this point in the history

  4. Add tests for loading user configuration 

    Signed-off-by: Krzesimir Nowak <krzesimir@kinvolk.io>
    @krnowak
    krnowak committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    bb7446a View commit details
    Browse the repository at this point in the history

  5. Simplify the serialize_config test 

    Setting up the Package struct in this test was necessary just to have
things to compile, otherwise nothing from the struct was used at
all. With the TestPkg struct in place, we don't need to do the setup
anymore.

Signed-off-by: Krzesimir Nowak <krzesimir@kinvolk.io>
    @krnowak
    krnowak committed Nov 3, 2017
    Configuration menu
    Copy the full SHA
    50f3aad View commit details
    Browse the repository at this point in the history