Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terraform ProfitBricks Builder #7943

Merged
merged 12 commits into from
Jan 18, 2017
Merged

Terraform ProfitBricks Builder #7943

merged 12 commits into from
Jan 18, 2017

Conversation

jasmingacic
Copy link

@jasmingacic jasmingacic commented Aug 3, 2016

This is terraform builder for ProfitBricks.

While writing unit tests I've ran on to a issue with a computed field primary_nic in profitbricks_server as suggested by the output this is a Terraform bug. I'm pasting the output here:

         Terraform Version: 0.7.0
                    Resource ID: profitbricks_loadbalancer.example
                    Mismatch reason: attribute mismatch: nic_id
                    Diff One (usually from plan): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"nic_id":*terraform.ResourceAttrDiff{Old:"", New:"${profitbricks_server.webserver.primary_nic}", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "name":*terraform.ResourceAttrDiff{Old:"", New:"loadbalancer", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "dhcp":*terraform.ResourceAttrDiff{Old:"", New:"true", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "datacenter_id":*terraform.ResourceAttrDiff{Old:"", New:"${profitbricks_datacenter.foobar.id}", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyTainted:false}
                    Diff Two (usually from apply): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"name":*terraform.ResourceAttrDiff{Old:"", New:"loadbalancer", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "dhcp":*terraform.ResourceAttrDiff{Old:"", New:"true", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "datacenter_id":*terraform.ResourceAttrDiff{Old:"", New:"0a1ea215-ab8a-4623-a40a-4d4f5bc7f249", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyTainted:false}

The property primary_nic is computed when a server is created this doesn't cause any problems when using the builder as a third-party builder. Also if id of profitbricks_nic is used this issue doesn't exist. It seems that apply doesn't pick up this property.

I also tried writing unit tests for import but I kept running into problems. For example if I'm to import a profitbricks_lan the datacenter_id on which profitbricks_lan depends, but datacenter_id value wouldn't be passed on to read function. Also some properties of other resources don't get recorded into tfstate.

Can you advise us on how to fix that?

profitbricks_loadbalancer resource doesn't function correctly due to a known issue with the REST API. It is due to be released in upcoming releases.

@jasmingacic
Copy link
Author

@jen20 @radeksimko Guys do you have any remarks?

@baldwinSPC
Copy link

@jen20 @radeksimko pinging on this PR.

@jasmingacic
Copy link
Author

@mwhooker Is there anything i should do here?

@jasmingacic
Copy link
Author

@mwhooker I'm checking in again

@jasmingacic
Copy link
Author

ping?

@mwhooker
Copy link
Contributor

looks like this needs a rebase

@jasmingacic
Copy link
Author

Check

Copy link
Contributor

@stack72 stack72 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jasminSPC

Thanks for all the work here - sorry this has taken so long to get back to! I have just given this a quick once through for some stylistic comments

I have left some inline that could be done with rolling across the entire provider

If you have it, please can you run the acceptance tests and add the output to the PR? :)

Thanks

Paul

func (c *Config) Client() (string, error) {
profitbricks.SetAuth(c.Username, c.Password)
profitbricks.SetDepth("5")
log.Printf("[DEBUG] Username and password %s : %s", c.Username, c.Password)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should log the password here

DefaultFunc: schema.EnvDefaultFunc("PROFITBRICKS_PASSWORD", nil),
Description: "Profitbricks password for API operations.",
},
"timeout": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we set a default timeout?

config := Config{
Username: d.Get("username").(string),
Password: d.Get("password").(string),
Timeout: d.Get("timeout").(int),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check that this is set otherwise, it will be a 0 being passed - is that an issue?

}

func resourceProfitBricksDatacenterCreate(d *schema.ResourceData, meta interface{}) error {
username, password, _ := getCredentials(meta)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this feels onerous for someone extending - I think there should be a helper func that takes care of this

conn := getProfitBricksConnection(meta)

}

resp := profitbricks.PatchDatacenter(d.Id(), obj)
waitTillProvisioned(meta, resp.Headers.Get("Location"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't we have to get Location from state here?

waitTillProvisioned(meta, resp.Headers.Get(d.Get("Location").(string)))

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ProfitBricks REST returns request status url in "Location" response header.
https://devops.profitbricks.com/api/cloud/v3/#how-to-access-the-api
image


return fmt.Errorf("Request failed with following error: %s", request.Metadata.Message)
}
time.Sleep(10 * time.Second)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we have to sleep here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While polling the request status every iteration we sleep for 10 seconds.

if err != nil {
return err
}
return nil //resourceProfitBricksFirewallRead(d, meta)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update should call Read func as a general rule to make sure all the state values get set as expected

resp := profitbricks.DeleteLan(d.Get("datacenter_id").(string), d.Id())
if resp.StatusCode > 299 {
//try again in 20 seconds
time.Sleep(60 * time.Second)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment above this says 20 seconds - can we clean that up - do we have to sleep?

d.Set("ip", lb.Properties.Ip)
d.Set("dhcp", lb.Properties.Dhcp)
if lb.Entities.Balancednics != nil && len(lb.Entities.Balancednics.Items) > 0 {
d.Set("nic_id", lb.Entities.Balancednics.Items[0].Id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we guarantee that the NIC at [0] is the correct one?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since "nic_id" is a required field it is not necessary to set it here. removed

@jasmingacic
Copy link
Author

I've addressed all remarks.
While doing that I also renamed "timeout" to "retries" as it is better description.

@jasmingacic jasmingacic reopened this Dec 20, 2016
@edevenport
Copy link

@jen20 @radeksimko @stack72 Is there anything else required to have this PR merged?

@stack72
Copy link
Contributor

stack72 commented Jan 13, 2017

Hi @jasminSPC

I will get this reviewed and merged next week! Please ping me if you haven't heard from me before Tuesday end of day :)

P.

@stack72 stack72 self-assigned this Jan 13, 2017
@jasmingacic
Copy link
Author

@stack72 Ping. It's end of the day where I live :)

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

@jasminSPC on this now :)

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

Hi @jasminSPC

This LGTM now - I will need to get a ProfitBricks account setup so that I can test this out. I will have that done today - when the folks on the US West Coast wake up :)

Thanks for rounding off those changes

Paul

@jasmingacic
Copy link
Author

@stack72 We've added three data sources to our third party plugin. Do you think it would be good to add it now.

As for the credentials we can provide you those. Can you just email me so I can send it over to you.

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

ok, let's follow up with the datasources as another PR - I don't want to delay this anymore :)

If you have creds for testing, can you email them to stack72 @ hashicorp.com

P.

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

Hi @jasminSPC

Ok, I have ran the tests and we are getting failures. The test run looks as follows:

% make testacc TEST=./builtin/providers/profitbricks                                                                                141 ↵ ✭
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/01/18 13:08:01 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/profitbricks -v  -timeout 120m
=== RUN   TestAccProfitBricksDatacenter_importBasic
--- PASS: TestAccProfitBricksDatacenter_importBasic (32.06s)
=== RUN   TestAccProfitBricksFirewall_importBasic
--- FAIL: TestAccProfitBricksFirewall_importBasic (445.24s)
	testing.go:265: Step 1 error: Resource specified by ResourceName couldn't be found: profitbricks_firewall.foobar
=== RUN   TestAccProfitBricksIPBlock_importBasic
--- FAIL: TestAccProfitBricksIPBlock_importBasic (0.45s)
	testing.go:265: Step 0 error: Error applying: 1 error(s) occurred:

		* profitbricks_ipblock.webserver_ip: An error occured while reserving an ip block: {
		  "httpStatus" : 422,
		  "messages" : [ {
		    "errorCode" : "100",
		    "message" : "[(root).properties.location] Location 'ipblock' does not exist"
		  } ]
		}

=== RUN   TestAccProfitBricksLan_importBasic
--- FAIL: TestAccProfitBricksLan_importBasic (73.86s)
	testing.go:265: Step 1 error: 1 error(s) occurred:

		* profitbricks_lan.webserver_lan: An error occured while fetching a lan ID 1 {
		  "httpStatus" : 404,
		  "messages" : [ {
		    "errorCode" : "309",
		    "message" : "Resource does not exist"
		  } ]
		}

=== RUN   TestAccProfitBricksLoadbalancer_importBasic
--- FAIL: TestAccProfitBricksLoadbalancer_importBasic (428.06s)
	testing.go:265: Step 0 error: After applying this step, the plan was not empty:

		DIFF:

		UPDATE: profitbricks_loadbalancer.example
		  ip: "10.11.226.225" => ""

		STATE:

		profitbricks_datacenter.foobar:
		  ID = e6d976a9-ccca-45d0-b403-059ab807390d
		  description =
		  location = us/las
		  name = loadbalancer-test
		profitbricks_loadbalancer.example:
		  ID = b22a7f19-89cb-450d-9d89-27f586923a24
		  datacenter_id = e6d976a9-ccca-45d0-b403-059ab807390d
		  dhcp = true
		  ip = 10.11.226.225
		  name = loadbalancer
		  nic_id = ae5e7264-9f16-431d-a613-382734e4b5c2

		  Dependencies:
		    profitbricks_datacenter.foobar
		    profitbricks_nic.database_nic
		profitbricks_nic.database_nic:
		  ID = ae5e7264-9f16-431d-a613-382734e4b5c2
		  datacenter_id = e6d976a9-ccca-45d0-b403-059ab807390d
		  dhcp = true
		  firewall_active = true
		  lan = 2
		  name = updated
		  server_id = 17ee3235-d635-4e72-8f9c-f2330620ab5d

		  Dependencies:
		    profitbricks_datacenter.foobar
		    profitbricks_server.webserver
		profitbricks_server.webserver:
		  ID = 17ee3235-d635-4e72-8f9c-f2330620ab5d
		  availability_zone = ZONE_1
		  boot_volume = 52d60b63-8823-4c76-b020-26f1dfc5e196
		  cores = 1
		  cpu_family = AMD_OPTERON
		  datacenter_id = e6d976a9-ccca-45d0-b403-059ab807390d
		  name = webserver
		  nic.# = 1
		  nic.1633388749.dhcp = true
		  nic.1633388749.firewall.# = 1
		  nic.1633388749.firewall.506939247.icmp_code =
		  nic.1633388749.firewall.506939247.icmp_type =
		  nic.1633388749.firewall.506939247.ip =
		  nic.1633388749.firewall.506939247.name = SSH
		  nic.1633388749.firewall.506939247.port_range_end = 22
		  nic.1633388749.firewall.506939247.port_range_start = 22
		  nic.1633388749.firewall.506939247.protocol = TCP
		  nic.1633388749.firewall.506939247.source_ip =
		  nic.1633388749.firewall.506939247.source_mac =
		  nic.1633388749.firewall.506939247.target_ip =
		  nic.1633388749.firewall_active = true
		  nic.1633388749.ip =
		  nic.1633388749.lan = 1
		  nic.1633388749.name =
		  primary_nic =
		  ram = 1024
		  volume.# = 1
		  volume.901867600.bus =
		  volume.901867600.disk_type = SSD
		  volume.901867600.image_name = ubuntu-16.04
		  volume.901867600.image_password = test1234
		  volume.901867600.licence_type =
		  volume.901867600.name = system
		  volume.901867600.size = 5
		  volume.901867600.ssh_key_path =

		  Dependencies:
		    profitbricks_datacenter.foobar
=== RUN   TestAccProfitBricksNic_importBasic

Can you have a look?

Paul

@stack72 stack72 added the waiting-response An issue/pull request is waiting for a response from the community label Jan 18, 2017
@jasmingacic
Copy link
Author

@stack72 we were actually planning to remove imports. Do you mind if I remove them now?

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

ok, let's remove import for now :) We can revisit that later

@jasmingacic
Copy link
Author

jasmingacic commented Jan 18, 2017

Give me a minute to wrap it up and sync the fork

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

👍 I am on this PR today - I promised :)

@jasmingacic
Copy link
Author

@stack72 There you go

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

thanks @jasminSPC

Testing again now!

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

Hi @jasminSPC

This LGTM! now :)

% make testacc TEST=./builtin/providers/profitbricks                                                                                                                                                  ✭
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/01/18 13:59:30 Generated command/internal_plugin_list.go
TF_ACC=1 go test ./builtin/providers/profitbricks -v  -timeout 120m
=== RUN   TestProvider
--- PASS: TestProvider (0.00s)
=== RUN   TestProvider_impl
--- PASS: TestProvider_impl (0.00s)
=== RUN   TestAccProfitBricksDataCenter_Basic
--- PASS: TestAccProfitBricksDataCenter_Basic (88.35s)
=== RUN   TestAccProfitBricksFirewall_Basic
--- PASS: TestAccProfitBricksFirewall_Basic (1067.70s)
=== RUN   TestAccProfitBricksIPBlock_Basic
--- PASS: TestAccProfitBricksIPBlock_Basic (23.99s)
=== RUN   TestAccProfitBricksLan_Basic
--- PASS: TestAccProfitBricksLan_Basic (153.79s)
=== RUN   TestAccProfitBricksLoadbalancer_Basic

@stack72
Copy link
Contributor

stack72 commented Jan 18, 2017

Thansk for all the work here :)

@stack72 stack72 merged commit 7e9c850 into hashicorp:master Jan 18, 2017
catsby added a commit that referenced this pull request Jan 19, 2017
* master: (232 commits)
  Update CHANGELOG.md
  command/plugin_list: Adding Alicloud to the plugin list file (#11292)
  Additionnal information for machine type (#11288)
  provider/alicloud: Creating the necessary structure for the alicloud documentation (#11289)
  Update CHANGELOG.md
  update alicloud provider (#11235)
  Update CHANGELOG.md
  provisioner/chef: Support named run-lists for Policyfiles (#11215)
  Correct data.aws_route_table filter AWS docs link
  Update CHANGELOG.md
  Update CHANGELOG.md
  removes region param from google_compute_backend_service (#10903)
  core: Add pathexpand interpolation function
  Update CHANGELOG.md
  Add instance_tags as an additional filter
  provider/aws: Add aws_instance data source
  Update `parameter_group_name` (#11269)
  provider/profitbricks: Rename the profitbricks bin so that the plugin (#11267)
  Update CHANGELOG.md
  Terraform ProfitBricks Builder (#7943)
  ...
@ghost
Copy link

ghost commented Apr 17, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement new-provider waiting-response An issue/pull request is waiting for a response from the community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants