HCL template

[FrenchBen]

As we've seen, there has been quite a few discussions around the use of a "template engine" or something similar in order to introduce more logic to the current template interpolation.
https://github.com/hashicorp/hcl2/blob/master/hcl/hclsyntax/spec.md#templates

I'd like to know the motivation in re-defining yet another template engine/syntax, when so many have been created before (and simply work) and could be re-used?

• Go's template: https://golang.org/pkg/text/template/
• Liquid's template: https://shopify.github.io/liquid/
• Jinja's template: http://jinja.pocoo.org/

[ghost]

Go's template engine would be amazing. Sent from mobile

…

On Mar 12, 2018 6:17 PM, "French Ben" ***@***.***> wrote: As we've seen, there has been quite a few discussions around the use of a "template engine" or something similar in order to introduce more logic to the current template interpolation. https://github.com/hashicorp/hcl2/blob/master/hcl/ hclsyntax/spec.md#templates I'd like to know the motivation in re-defining yet another template engine/syntax, when so many have been created before (and simply work) and could be re-used? - Go's template: https://golang.org/pkg/text/template/ - Liquid's template: https://shopify.github.io/liquid/ - Jinja's template: http://jinja.pocoo.org/ — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#25>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Ad1pYYn3EVQnTMqHF_xiq_pgrpdN51I2ks5tdx49gaJpZM4Sn7zE> .

[apparentlymart]

The constraint here is that what we define must be compatible with existing syntax in HIL. The new syntax is therefore designed to be a superset of that existing syntax so that existing Terraform configurations can work with little or no change. The interpolation syntax is used in almost every Terraform configuration and illustrated in examples all over the web and in printed books, so changing it at this point in Terraform's life is not reasonable.

The other template languages mentioned here are not compatible with the HIL interpolation syntax, so while indeed we would've liked to use one of these if we had the luxury of starting from scratch, language design is often a matter of compromises. The HCL template syntax here takes inspiration from these other languages, but must also take inspiration from HIL.

Syntax issues aside, it's also important to note that HCL's type system is not exactly compatible with Go's (it includes the idea of "unknown" values -- or <computed> in Terraform -- and has structural rather than named types) and so even if we did build on Go's Template syntax it would not be possible to re-use the existing implementation in text/template. Jinja was actually the more likely one to use, since the syntax has proven itself compatible with the type systems of a number of languages by now (e.g. in Nunjucks), but due to the syntax constraints mentioned above we instead went for mimicking some of its features within the existing precedent set by HIL.

[FrenchBen]

@apparentlymart I understand some of the constraints/complexity listed above, yet it seems that the constraint itself has been established for no other reason than to keep everything assigned to 1 attribute: template.
If we allow ourselves to slightly diverge from this, and introduce a template_jinja attribute, then everything becomes much simpler, albeit we would need to document the incompatibility in having both.
If this seems like duplicate effort, a similar solution was suggested by you in another thread, in order to keep backwards compatibility, and prevent Golang type conflicts:
hashicorp/terraform-provider-tls#3 (comment)
organizational_unit vs organizational_units

Another approach, is to check the file extensions passed to the template attribute, and enforce something like a .jinja .jinja2 .j2 file extension for it to be parsed as a Jinja template.
This is a pretty common pattern in ansible, along with support of <computed> values:
http://docs.ansible.com/ansible/latest/template_module.html

[apparentlymart]

@FrenchBen, I think you are discussing Terraform's template_file data source here, rather than HCL2 itself.

It is possible that it could support other template engines in future, but that would be a matter to discuss it that provider's own repository, rather than here, since it would be a change to the implementation of that provider rather than to the HCL2 parser and interpreter.

My comment above was discussing the reason why HCL2 defines templates the way it does; this is a separate question from what template engine(s) the template_file data source might use, and there's no technical reason why that data source couldn't (as you suggest) support multiple engines. Ultimately that's a matter of weighing the benefit of that flexibility against the cost of implementing and maintaining the interfaces to multiple template engines, particularly if some of those engines are written in a language other than Go.

The possibility of multiple template engines was discussed in hashicorp/terraform-provider-template#1. Nobody on the Terraform team is working on that right now because we're all busy working on other things, but that issue is a good place to discuss the implementation tradeoffs here and figure out what makes sense to implement.

[FrenchBen]

@apparentlymart understood - I ended up on the section around HCL2, from multiple github searches and the need to do more with templates. It was not my intention to steer the focus in the wrong direction, but rather provide a greater level of integration with known template formats, in order to provide greater flexibility for deploying infrastructure.

Within the current docs, templates are mentioned as part of the HCL interpolation, and not as a separate provider:
https://www.terraform.io/docs/configuration/interpolation.html#templates

The confusion came from the correlation of string interpolation under the label of template sub-language; there is no template actually used here, but rather a pattern for variable substitution and the supported substitution interpretation.

[apparentlymart]

Since hashicorp/terraform-provider-template#1 is already covering the idea of having Terraform's template_file data source support other template engines, and that's outside of HCL's scope (HCL's integrated template syntax will be the only one supported at the parser level), I'm going to close this just to consolidate the discussion over in that other issue.

Terraform's configuration language docs will also get an overhaul as part of preparing for its first release containing the new HCL implementation (which will be v0.12) so along the way we can try to make the relationship between the template language and the template provider a little clearer.

Thanks for bringing this up!