nomad uses off-release hcl2 version

[notnoop]

Nomad uses an off-release hcl2 library: https://github.com/hashicorp/hcl/tree/hcl2-nomad-teaks . It branches off the main hcl2 branch, with few PRs applied: hashicorp/hcl#414, hashicorp/hcl#413, hashicorp/hcl#413 .

Context and Why

The changes primary target the gohcl, a helper package that evaluates and decodes hcl AST tree into Nomad. The library is well suited for small simple structures, and other HashiCorp use it in very limited way. However, for the complicated config structures, other HashiCorp products use different approaches: Packer uses a custom code generation framework using the hcldec library (see https://github.com/hashicorp/packer/tree/v1.6.6/hcl2template ). Terraform uses a hierarchical approach: terraform decodes the structure manually, but uses gohcl for evaluating the fine-grain attribute expressions (see https://github.com/hashicorp/terraform/blob/v0.14.5/configs/module_call.go ).

Nomad opted to push as much parsing logic into gohcl and rely on hcl: field attributes, rather than maintain hand-rolled or code generated decoders. Also, we opted to reuse the API public struct, api.Job, for target parsing: to eliminate the overhead to maintain two different projects of the config file, we frequently had with the jobspec parser; the downside is the complexity of compatibility maintenance (don't want to change the api struct fields) and avoiding inserting hclv2 dependency into the nomad api package.

The PRs above fills the gap that Nomad needs to address their needs. Understandably, these are Nomad specific requirements and don't represent the common case of gohcl users. However, I believe it's still cheaper to maintain this small fork with relatively small diff than roll a custom full decoder like other projects have gone.

Next Steps

We should aim to get these PRs across the line and merged upstream, taking the hcl2 maintainers feedback and making them useful for others. Alternatively, we should re-examine our approach so that we implement the logic outside the hcl2 library: we can maintain our own gohcl package and/or adopt Terraform/Packer approaches of reserving gohcl to small structs. Also, we can add a new job struct meant for parsing with upstream gohcl library. then convert it to api.Job.