Improve Terraform Plugin

[chungers]

A few ideas:

• Currently the schema of the plugin accepts only one resource (i.e. one type field per struct). This makes it hard to specify a list of related resources necessary for the provisioning of the compute instance. For example, an instance could have a per-node SSH key or cert that need to be created and associated to the instance. The ability to specify a list of resources in the schema would make it possible to have terraform create them in one step.
• Make it possible to override the terraform command and provide environment variables to the command.
• Implement the metadata SPI to expose instance details so that properties like IP addresses can be read via infrakit metadata cat _path_

@JacobFrericks
@ndegory
@linsun

[JacobFrericks]

It would also be nice to have the ability to specify hostnames/datacenters/counts via terraform variables. Right now those are either hard coded in Infrakit's group.json, or auto-generated by Infrakit.

[chungers]

@JacobFrericks - can you give me an example of your current config where these variables are hard coded? Just want to get an idea what a typical input might look like. Thanks.

[JacobFrericks]

Here is my group.json:

{
  "ID": "terraform_demo_swarm_worker_sl",
  "Properties": {
    "Allocation": {
      "Size": 1
    },
    "Instance": {
      "Plugin": "instance-terraform",
      "Properties": {
        "type": "softlayer_virtual_guest",
        "value": {
          "count": "1",
          "hostname": "hardcoded_hostname",
          "domain": "hardcoded_domain",
          "os_reference_code": "UBUNTU_LATEST",
          "datacenter": "dal10",
          "cores": 1,
          "memory": 2048,
          "local_disk": true,
          "hourly_billing": true,
          "user_metadata": "apt-get update -y\napt-get upgrade -y\nwget -qO- https://get.docker.com/ | sh\n"
        }
      }
    },
    "Flavor": {
      "Plugin": "flavor-swarm/worker",
      "Properties": {
        "DockerRestartCommand": "service docker restart",
        "SwarmJoinIP": "<ip>",
        "Docker" : {
          "Host" : "<host>"
        }
      }
    }
  }
}

Terraform handles variables like this:

"hostname": "${var.hostname}",

Where you have a variable file that defines the variable name (ex: hostname) and the default value if the variable is missing. I'm not sure if you could do something similar or not.

[JacobFrericks]

Another input would be running the terraform apply command with the -no-color flag. This cleans up some of the output. It would go from:

level=info msg="\x1b[0m\x1b[1msoftlayer_virtual_guest.instance-1487790097: Still creating... (2m10s elapsed)\x1b[0m\x1b[0m\n" terraform=apply 

to:

level=info msg="softlayer_virtual_guest.instance-1487790097: Still creating... (2m10s elapsed)\n" terraform=apply 

[ndegory]

@chungers, focusing on AWS, I'd also like to be able to create on a per-instance basis:

• EBS volumes
• Elastic IPs
  and attach them to the instance.

[kaufers]

We noticed an issue with verbose terraform logging is applied.

When we set TF_LOG=TRACE then the terraform apply generates a lot of output. It seems that the reader here at https://github.com/docker/infrakit/blob/master/examples/instance/terraform/plugin.go#L213 is unable to process the output by line -- when I recreated it manually the longest line was 15,123 characters long.

I suspect that the buffer is full before the \n is encountered; this is causing the apply process to apparently never complete. The Infrakit logs keep showing that the resource is Still creating ....

[kaufers]

Terraform will not be able to remove the last member of a group because it requires at least a single .tf.json file. See https://groups.google.com/forum/#!topic/terraform-tool/14YB-9JEfXs

As a workaround we can place a file like empty.tf.json with {}; having this file allow the resource associated with the last .tf.json file to be removed.