Template debugging tool

[christophermaier]

Currently, it's difficult to know if templated content will behave as you expect until it's actually rendered by a live Supervisor. Unfortunately, this can mean unnecessarily long development and debugging cycles, and increased likelihood of escaped bugs.

In keeping with the general Habitat principle of pushing failure as close to the development phase as possible, it would be great to have a tool that could evaluate template files in a number of ways.

• Are variables being used actually available in the render context?
• Is valid Handlebars syntax being used?
• Are only recognized helpers being used?
• Will the rendered file be syntactically valid? This could be tricky, as this can mean different things depending on that kind of file is being templated (shell script, TOML file, JSON file, etc.)

Additionally, it would be nice to have a way to render the file using different input data to ensure it is working generally as expected.

[qubitrenegade]

I mentioned this in #1779 but I vote to name the linter tool roomba so I can hab studio roomba

[qubitrenegade]

Not sure if this fits in here or would belong another place, but it would be really cool to be able to stub or fake the gossip data.

E.g.: I've been working on a "consul-client" plan which would bind to a "core/consul" cluster. The only way to get that bind info to resolve is to actually build a 3node cluster then run my client plan against it.

I would like to be able to see that this:

  {{~#eachAlive bind.consul-server.members as |member|}}
    "{{member.sys.ip}}"{{~#unless @last}},{{/unless}}
  {{~/eachAlive}}

Actually resolves in the proper format, etc. (I'm still not clear on where to use the ~ at the beginning or end of a handlebars statement, so the only way is to change it, rebuild the plan, test it, change it rebuild the plan, test it, etc.)

[jsirex]

I want to raise attention to that topic and drop few notes on how it might be implemented.
I believe this also can resolve: #1137, #1779, #1677.
I don't know Rust so I'm kindly asking maintainers or community to help here as this topic is crucial for habitat services development.

Could you please implement hab-plan-render command line tool.
ARGS: takes only one:

• PATH - the path to a plan/directory, should behave same as hab-plan-builds' PATH arg

FLAGS:

• --input-toml - path to "input" toml. Defaults to $PLAN_CONTEXT/default.toml
• --extra-toml - path to additional toml files. Can be repeated multiple times. Merges in occur order
• --output - where to put rendered files. Defaults to $PLAN_CONTEXT/results

Exit code:

• 0 - success
• non-zero - any codes for file not found, render issues, etc.
• STDERR with explanations.

Code of this tool can easily be based on hab-sups' code, so implementation should be easy.

Usage example:

1. I create tests/user.toml - emulate customization by changing default settings somehow

port=9999 # overridden port

2. I create tests/svc.toml - emulate bindings, sys and other stuff I'm going to test

[sys]
ip = "100.100.100.100"
[bind]
[bind.database]
port = 5432
[bind.database.superuser]
name = "not_a_admin"
password = "not_a_password"
...

3. Now I can do write-check iterations with the single command:

hab-plan-render --extra-toml tests/svc.toml .
cat results/hooks/run # see what I got
grep 'db.admin = "not_a_admin" results/config/db.conf || echo "db.admin was not rendered!"

# OR INSPEC TEST
inspec inspec/template_contains_required_values_spec.rb

[christophermaier]

I commented on this in Slack, but will leave it here, too:

I imagine it'd end up looking something like that, yes. Unfortunately, the core team isn't going to have bandwidth to get to that for a while. If anyone in the community wants to take a stab at a prototype, that'd be cool, though.

[qubitrenegade]

@christophermaier I'd like to take a crack at it!

Any advice on where to start? E.g.: how does "render template" function work in "the real world"?

[christophermaier]

@qubitrenegade Looks like our configuration logic is a specialization of our hook compilation logic. This work may reveal additional opportunities for refactoring, consolidation, etc.

The configuration logic lives in:

habitat/components/common/src/templating/config.rs

Lines 464 to 536 in 6784490

	pub fn compile<P, T>(
	&self,
	service_group_name: &str,
	pkg: &Pkg,
	render_path: P,
	ctx: &T,
	) -> Result<bool>
	where
	P: AsRef<Path>,
	T: Serialize,
	{
	// JW TODO: This function is loaded with IO errors that will be converted a Supervisor
	// error resulting in the end-user not knowing what the fuck happned at all. We need to go
	// through this and pipe the service group through to let people know which service is
	// having issues and be more descriptive about what happened.
	let mut changed = false;
	for template in self.0.get_templates().keys() {
	let compiled = self.0.render(&template, ctx)?;
	let compiled_hash = crypto::hash::hash_string(&compiled);
	let cfg_dest = render_path.as_ref().join(&template);
	let file_hash = match crypto::hash::hash_file(&cfg_dest) {
	Ok(file_hash) => file_hash,
	Err(e) => {
	debug!("Cannot read the file in order to hash it: {}", e);
	String::new()
	}
	};
	if file_hash.is_empty() {
	debug!(
	"Configuration {} does not exist; restarting",
	cfg_dest.display()
	);
	let mut config_file = File::create(&cfg_dest)?;
	config_file.write_all(&compiled.into_bytes())?;
	outputln!(
	preamble service_group_name,
	"Created configuration file {}",
	cfg_dest.display()
	);
	set_permissions(&cfg_dest, pkg)?;
	changed = true
	} else if file_hash == compiled_hash {
	debug!(
	"Configuration {} {} has not changed; not restarting.",
	cfg_dest.display(),
	file_hash
	);
	continue;
	} else {
	debug!(
	"Configuration {} has changed; restarting",
	cfg_dest.display()
	);
	outputln!(
	preamble service_group_name,
	"Modified configuration content in {}",
	cfg_dest.display()
	);
	let mut config_file = File::create(&cfg_dest)?;
	config_file.write_all(&compiled.into_bytes())?;
	set_permissions(&cfg_dest, pkg)?;
	changed = true;
	}
	}
	Ok(changed)
	}
	}

The templating logic lives in:

habitat/components/common/src/templating/mod.rs

Lines 82 to 92 in 6784490

	pub fn render<T>(&self, template: &str, ctx: &T) -> Result<String>
	where
	T: Serialize,
	{
	let raw = serde_json::to_value(ctx).map_err(Error::RenderContextSerialization)?;
	debug!("Rendering template with context, {}, {}", template, raw);
	self.0
	.render(template, &raw)
	.map_err(|e| Error::TemplateRenderError(format!("{}", e)))
	}
	}

The key bit is that both take a "context", which is anything that implements serde's Serialize, which should come in quite handy here. In the Supervisor, we assemble a RenderContext, which is basically all the data you have available to you in a template. The structure in the Supervisor is a little complex, but all the rendering logic cares about is that it's something that you can basically treat like a big JSON object (the schema of which is defined here. You could create a HashMap of information and pass it into the rendering logic and it should be happy with that. You could also easily take in one or more JSON/TOML files and merge them all together to generate a "context"; the fact that it just needs to be something that implements Serialize means you've got lots of options.

Hopefully that gives you a better idea of how it all gets put together; let me know if you've got any other questions. And thanks for the help!

[qubitrenegade]

I opened PR #6114 as I figured it would be easier to discuss changes.

So first thing I did was wire up a hab plan render subcommand. That seemed like the most appropriate place to put it... I had also considered a hab plan debug render which would provide the opportunity for other commands. I'm certainly open to changing it!

I figured it would be easier to render one file and then I can figure out how to render a whole directory.

As you might infer from my variable names I was thinking that you'd want to maybe pass the path to a default.toml and some kind of "mock_data.json" file that could mock out the sys.members data structures.

So I kinda cheated and just copied src/command/plan/init.rs to src/command/plan/render.rs. Then I used the serde_json method to slurp a json file and render the template! Neat!

Unfortunately, the way the init renders the template is directly with the Handlebars library, so templates that make use of helpers can't be rendered.

So I attempted to use the use crate::common::templating::TemplateRender crate...

I did this:

    let mut renderer = TemplateRenderer::new();
    renderer
        .register_template_string(&template_path, &template)
        .expect("Could not register template content");
    let rendered_template = renderer.render(&template_path, &json).ok().unwrap();

Which I'm not sure I entirely understand, but I think does what I expect... sort of...

For instance, testing the consul config template with this test.json file:

{
  "cfg": {
    "server": {
      "datacenter": "fooo"
    },
    "ports": {
      "dns": 8000,
      "http": 8500,
      "https": 8501
    }
  },
  "svc": {
    "members": [
      { "sys": { "ip": "1.1.1.1" }},
      { "sys": { "ip": "2.2.2.2" }},
      { "sys": { "ip": "3.3.3.3" }}
    ]
  }
}

I only get:

{
  "datacenter": "fooo",
  "data_dir": "",
  "log_level": "",
  "bind_addr": "",
  "client_addr": "",
  "server": ,
  "retry_join": [
  ],
  "ports": {
    "dns": 8000,
    "http": 8500,
    "https": 8501,
    "serf_lan": ,
    "serf_wan": ,
    "server": 
  }
}

So I need some way to mock out the running service members.

Any thoughts?

Also before I get too far, any thoughts on the path I'm going down?/Feedback on code?

To test, I did the following:

cd ~/habitat/components/hab/
mkdir test
wget --directory-prefix test/ https://raw.githubusercontent.com/habitat-sh/core-plans/master/consul/config/basic_config.json
cat <<EOF | tee test/test.json
{
  "cfg": {
    "server": {
      "datacenter": "fooo"
    },
    "ports": {
      "dns": 8000,
      "http": 8500,
      "https": 8501
    }
  },
  "svc": {
    "members": [
      { "sys": { "ip": "1.1.1.1" }},
      { "sys": { "ip": "2.2.2.2" }},
      { "sys": { "ip": "3.3.3.3" }}
    ]
  }
}
EOF

cargo run -- plan render test/basic_config.json --mock-data test/test.json --render-dir result/config --print

Thanks!

[qubitrenegade]

Ok, a bit clumsy... but try:

cd ~/habitat/components/hab

Then: -

cargo run -- plan render ../../test/fixtures/render/consul/config/consul_config.json \
  --default-toml ../../test/fixtures/render/consul/default.toml \
  --user-toml ../../test/fixtures/render/consul/user.toml \
  --mock-data ../../test/fixtures/render/consul/override.json \
  --render-dir result/config \
  --print

or

cargo run -- plan render ../../test/fixtures/render/consul/hooks/run \
  --default-toml ../../test/fixtures/render/consul/default.toml \
  --user-toml ../../test/fixtures/render/consul/user.toml \
  --mock-data ../../test/fixtures/render/consul/override.json \
  --render-dir result/hooks \
  --print

This should render:

{
  "datacenter": "IN_OVERRIDE_JSON",
  "data_dir": "IN_DEFAULT_TOML",
  "log_level": "IN_USER_TOML",
  "bind_addr": "9.9.9.9",
  "client_addr": "9.9.9.9",
  "server": true,
  "retry_join": [
  ],
  "ports": {
    "dns": 6666,
    "http": 6667,
    "https": 6668,
    "serf_lan": 8888,
    "serf_wan": 8302,
    "server": 9999
  }
}

Still haven't figured out how to render for eachAlive. and need to figure out how to load pkg. data...

TODO:

• Figure out how to load svc data for eachAlive helper
• figure out how to load pkg. data. e.g. for {{pkg.svc_config_path}}
• started taking some notes here.

[qubitrenegade]

Still haven't figured out how to render for eachAlive

Helps if you use "alive" not "active" 🦆

https://github.com/qubitrenegade/habitat-1/blob/72d9ce5e429e6e6721f8c7ac0c35f1da0a035c34/test/fixtures/render/consul/override.json#L17-L22

[baumanj]

Now that #6114 is merged, can we close this, or is there more to do?

[qubitrenegade]

Well...

• Plan Linting: More obvious failures when "Habitat Helpers" fail. #5183
• hab plan render should load user data #6204
• hab plan render should take multiple --mock-data Parameters #6205
• hab plan render should use render_context_schema when rendering templates #6206
• hab plan render should provide better error messages when input template has errors #6279

Is my current "todo" list 😁

#6114 is really only a starting point or "MVP"... there's still a bunch I'd like to do (if you're willing to continue to hand-hold me?).

So... Maybe it makes sense to have a forum post and gather requirements and generate an overall "feature request list" and generate issues/requirements from that? I think we have a good list of requirements going so far, but that's largely based on how I use Habitat (and how I want to be able to debug habitat...).

Is there a way to "assign" those tickets to me? Then I don't have to keep a list of them?

Thanks!

[jsirex]

Is this ticket can be closed because we have hab plan render?

[christophermaier]

@jsirex Yup, looks like it... thanks for the nudge.