DO NOT MERGE: Habitize the Chef Server #1416

markan · 2017-10-27T23:51:58Z

#1416 cThis is a total WIP; and by no means is this ready to merge. It probably will change a lot as we work on things.

This represents a experimental take on what a chef-server might look like under habitat.

We're doing separate containers for each service, and starting them under control of a docker compose file. This is convenient for development, but totally sidesteps a whole bunch of operational questions.

Trying it out

Build it

hab studio enter
cd src
for dir in oc-id openresty-noroot nginx dbdpg bookshelf chef-server-ctl oc_bifrost oc_erchef; do cd $dir; build && hab pkg export docker $(ls -1t results/*.hart | head -1) ; cd ..; done

Running it (first time)

export HAB_ORIGIN=my_hab_origin
docker-compose up

Testing it

docker-compose exec chef-server-ctl chef-server-test
curl -k https://localhost/_status

Running it (subsequent times with empty data)

docker-compose down && docker system prune --volumes -f && docker-compose up

Note: the removal of volumes should not be needed once we make data bootstrapping convergent

TODO list:

oc_bifrost

oc_erchef

bookshelf

nginx

set up binds & dependencies
convert configuration
health check hook
remove pkg_svc_user=root requirement

chef-server-ctl

import omnibus-ctl, private-chef-ctl-commands & oc-chef-pedant to reproduce a chef-server-ctl admin experience
convert configuration
secrets generation & sharing via ring

stretch goals:

test docker run requirements: --network host --user notroot:notroot --cap-drop NET_BIND_SERVICE --cap-drop SETUID --cap-drop SETGID
mover??
day 2 operations (monitoring, secrets rekey, deployment/topology guidance)
public builder service to continuously integrate?

before merge

Rebase and Squash all commits (DCO violations)
Create/use an "official" habitat origin and dockerhub user

irvingpop · 2017-11-03T01:46:07Z

src/nginx/habitat/config/chef_https_lb.conf

@@ -6,14 +6,14 @@

    access_log {{pkg.svc_var_path}}/log/access.log opscode;
      ssl on;
-      ssl_certificate {{pkg.svc_data_path}}/ca/api.chef-server.dev.crt;
-      ssl_certificate_key {{pkg.svc_data_path}}/ca/api.chef-server.dev.key;
+      ssl_certificate {{pkg.svc_data_path}}/ca/{{cfg.server_name}.cert;


hey @markan startup failures are likely do to a missing second right brace here

stevendanna

🎉 ❤️ 💪

Great work so far! Thanks for digging into this. As I mentioned in the comments, I'd encourage those of you who worked on this to consider adding your names to the MAINTAINERS file.

I did an initial read through of this and left inline comments. Overall this looks like the right direction and is much less duplication than I was originally worried about.

I know this is still WIP so hopefully I wasn't /too/ annoying with mentioning things you probably already know about. I assume you'll call for reviewers when this is ready to go, but I was procrastinating so I decided to read it now :D.

Here are some larger things I would like us to consider doing. Some (most?) could happen post-merge as follow up work. These are roughly ordered by how strongly I feel about them.

Document what we intend to support in this setup. I'm thinking about: Add-ons, our large number of configurables in the reconfigure cookboks, etc.
Document basic configuration and operation. The documentation can mark this as experimental, but we should get the docs started soon to avoid getting too far behind.
Move chef-server-ctl to an app-bundled application and out of omnibus/files. This may be what you mean by "import" in your current todo list.
A gather-logs style command for producing a tarball for support with relevant info.
Review the lua routing code. My sense is that a huge portion of it could be ripped out. This would reduce a lot of the duplication this change introduces and simplify our lives to boot.
Take the secrets bootstrapping and erchef data bootstrapping commands you've written here, move them some place where we can share them with the old syle config and use them exclusively.
Some sort of travis check to remind us that we now have 2 sets of configuration files. I think keeping the habitat configs separate is the best way forward for now since that allows us to be purposeful in what configuration we support; however, we don't want it to drift so something that reminds us to check for that would be great.
A chef_secrets provider that can read directly from the hab ring to avoid passwords in configuration files.

In order to merge this, I think we need to finish (or have dedicated resources scheduled to finish):

[ ] Integration into the build & test pipeline
[ ] Documentation on at least:

installation,
upgrade (it is OK if we don't know how to handle migrations yet, we can document that once we have a migration to run)
data backup

stevendanna · 2017-11-10T13:04:59Z

docker-compose.yml

+    links:
+      - postgresql
+    environment:
+      - FOO=bar


What's this for? I see it a few times below as well.

This is what happens when you order the "Never Ending" Copy-Pasta Bowl at the Olive Garden 😂

stevendanna · 2017-11-10T13:06:04Z

docker-compose.yml

+
+volumes:
+  postgresql-data:
+  erchef-data:


No volume for the ES data?

stevendanna · 2017-11-10T13:08:36Z

src/bookshelf/habitat/config/database-migrations.sh

+#!/bin/bash
+
+# TODO: not sure how to handle this. Sqitch bombs when it can't find the timezone
+export TZ="America/Chicago"


If we do have to hard code this, maybe set it to UTC.

stevendanna · 2017-11-10T13:10:48Z

src/bookshelf/habitat/config/database-migrations.sh

@@ -0,0 +1,40 @@
+#!/bin/bash


Can we set -e here and in our other shell scripts?

stevendanna · 2017-11-10T13:11:24Z

src/bookshelf/habitat/config/sys.config

+ {kernel, [{inet_dist_use_interface, {127,0,0,1}}]},
+ %% SASL config
+ {sasl, [
+         {sasl_error_logger, {file, "{{pkg.svc_var_path}}//logs/sasl-error.log"}},


[nit] Double / here and in other places throughout the file.

stevendanna · 2017-11-10T14:17:36Z

src/oc_erchef/habitat/config/dark_launch_features.json

@@ -0,0 +1,16 @@
+{


Post-Merge TODO: We should really rip nearly all of these darklaunch flags out. Basically none of them wold work if changed.

stevendanna · 2017-11-10T14:18:20Z

src/oc_erchef/habitat/config/elasticsearch-index-init.json

@@ -0,0 +1,57 @@
+{


Can we move this to a location inside the erchef build so we can easily access it from the old code and the new code?

stevendanna · 2017-11-10T14:18:43Z

src/oc_erchef/habitat/config/private-chef-secrets.json

@@ -0,0 +1 @@
+{}


What needs this still?

stevendanna · 2017-11-10T14:20:22Z

src/oc_erchef/habitat/config/sys.config

+        %% be part of a loaded application
+        {node_license, 999999},
+        {upgrade_url, <<"http://www.chef.io/contact/on-premises-simple">>},
+        {actions_host, "127.0.0.1"},


I think we want actions permanently off int his setup so maybe we can remove all this stub config?

stevendanna · 2017-11-10T14:21:14Z

src/oc_erchef/habitat/default.toml

@@ -0,0 +1,98 @@
+[private_chef]


Similar to the other configs, we should check these for unused variables.

Signed-off-by: Elliott Davis <edavis@chef.io>

The build of this works but the config is still in progress. Signed-off-by: Elliott Davis <edavis@chef.io>