Partial updates on Ansible, Puppet and Chef #373

rambleraptor · 2018-07-27T19:39:16Z

This has been too long coming.

How does Terraform do integration tests on updates? I can do a manual test to verify that everything doesn't crash + burn, but I'd like something more automated in the future.

[all]

Partial updates on A/C/P

[terraform]

[puppet]

Partial updates

[puppet-compute]

[puppet-container]

[puppet-dns]

[puppet-logging]

[puppet-pubsub]

[puppet-resourcemanager]

[puppet-sql]

[puppet-storage]

[chef]

Partial updates

[chef-compute]

[chef-container]

[chef-dns]

[chef-logging]

[chef-sql]

[chef-storage]

[ansible]

Partial updates

modular-magician · 2018-07-27T20:09:54Z

I am a robot that works on MagicModules PRs!

I built this PR into one or more PRs on other repositories, and when those are closed, this PR will also be merged and closed.

modular-magician · 2018-07-27T20:55:05Z

I am (still) a robot that works on MagicModules PRs!

I just wanted to let you know that your changes (as of commit 1497538) have been included in your existing downstream PRs.

SirGitsalot · 2018-07-27T21:42:15Z

Is there any easy way to see the generated code for a PR?

rambleraptor · 2018-07-27T21:59:01Z

After the magician runs, it'll give a list of PRs with the generated code. If you look up a couple comments, you'll see the Magician message with a link to (link redacted)

rambleraptor · 2018-07-27T22:02:45Z

I added in Puppet support for partial updates too.

I started on Chef, but it seems like it'll be a lot trickier because we don't have an easy way to do property diffs.

danawillow · 2018-07-27T22:40:36Z

Here's an example of a Terraform test that tests update: https://github.com/terraform-providers/terraform-provider-google/blob/master/google/resource_compute_disk_test.go#L247. The Terraform libraries give us the ability to have multiple TestSteps in a given test, so in this case, it runs the basic config (https://github.com/terraform-providers/terraform-provider-google/blob/master/google/resource_compute_disk_test.go#L601) and then runs terraform as if you had changed the config to the updated one (https://github.com/terraform-providers/terraform-provider-google/blob/master/google/resource_compute_disk_test.go#L629). We don't have any way in Terraform to test that it was indeed an update and not a destroy/create, but this at least lets us check that the resource has the properties we expect at the end of the test.

rambleraptor · 2018-07-31T22:50:40Z

templates/puppetchef/update_props.erb

@@ -0,0 +1,23 @@
+<% update_props = properties_by_custom_update(object.all_user_properties) -%>


I'd like to start putting the shared Puppet+Chef code inside a puppetchef folder to clean up the templates/ directory. Any objections?

nat-henderson · 2018-08-07T01:13:50Z

api/type.rb

    # Represents a fingerprint.  A fingerprint is an output-only
    # field used for optimistic locking during updates.
-    class Fingerprint < String
+    class Fingerprint < FetchedExternal


Hm, what does that do to the Terraform output code?

This shouldn't do anything.

modular-magician · 2018-08-10T21:19:19Z

I am (still) a robot that works on MagicModules PRs!

I just wanted to let you know that your changes (as of commit ee93eca) have been included in your existing downstream PRs.

modular-magician · 2018-08-10T21:47:00Z

I am a robot that works on MagicModules PRs!

I built this PR into one or more PRs on other repositories, and when those are closed, this PR will also be merged and closed.
depends: GoogleCloudPlatform/puppet-google-compute#83
depends: GoogleCloudPlatform/chef-google-compute#45
depends: modular-magician/ansible#63

modular-magician · 2018-08-20T18:23:35Z

I am (still) a robot that works on MagicModules PRs!

I just wanted to let you know that your changes (as of commit 5045044) have been included in your existing downstream PRs.

Tracked submodules are build/puppet/_bundle build/puppet/auth build/puppet/bigquery build/puppet/compute build/puppet/sql build/puppet/storage build/puppet/spanner build/puppet/container build/puppet/dns build/puppet/pubsub build/puppet/resourcemanager build/chef/_bundle build/chef/auth build/chef/compute build/chef/sql build/chef/storage build/chef/spanner build/chef/container build/chef/dns build/chef/iam build/terraform build/ansible.

nat-henderson · 2018-08-20T20:38:34Z

Woop woop! This is huge. :)

googlebot added the cla: yes label Jul 27, 2018

modular-magician added the downstream-generated label Jul 27, 2018

rambleraptor requested review from nat-henderson, danawillow and SirGitsalot July 27, 2018 20:53

modular-magician added the downstream-generated label Jul 27, 2018

rambleraptor changed the title ~~Partial updates on Ansible~~ Partial updates on Ansible + Puppet Jul 27, 2018

rambleraptor force-pushed the partial-update branch from 16367ce to 45c1417 Compare July 31, 2018 22:46

rambleraptor changed the title ~~Partial updates on Ansible + Puppet~~ Partial updates on Ansible, Puppet and Chef Jul 31, 2018

rambleraptor commented Jul 31, 2018

View reviewed changes

rambleraptor force-pushed the partial-update branch from 6dee6ce to 00260e5 Compare July 31, 2018 23:03

nat-henderson reviewed Aug 7, 2018

View reviewed changes

rambleraptor force-pushed the partial-update branch 4 times, most recently from 0053480 to d470196 Compare August 10, 2018 20:09

rambleraptor removed request for danawillow and SirGitsalot August 10, 2018 20:37

modular-magician added the downstream-generated label Aug 10, 2018

rambleraptor removed the downstream-generated label Aug 10, 2018

rambleraptor force-pushed the partial-update branch from 49115f2 to 1633e6e Compare August 10, 2018 21:22

modular-magician added the downstream-generated label Aug 10, 2018

rambleraptor added 23 commits August 20, 2018 10:33

Chef seems to work

b4a9b20

tests are a real joy

dbbbd1e

rebase issues because object change

5157edf

trying to undo some API changes

5e263c9

build_url problems

a681ac8

bad formatting on update_fields

157f9ba

Ansible FetchedExternal support + adding labelFingerprint

2a21d72

test issues

e74ff20

indentation

4834ae3

ansible does updates based on input

9420548

better updates on puppet w/ input support

6feedbe

chef updates based on input

cbb2727

chef works with disk size change + label tests

3575eca

puppet works with disk size + labels update

0f97ac8

Ansible works

bbdb6c5

works with linter

675bbfd

removing old comments on metadata helper

abd7252

post_create on P/C

043058f

Reorganizing ansible into functions

93dc0b9

Ansible post create + vpn tunnel

79c2a3b

Ansible works

799384c

removing tests from puppet + chef

16a977d

changing to underscore

413c298

rambleraptor force-pushed the partial-update branch from 3da41a3 to 413c298 Compare August 20, 2018 17:54

Merge 413c298 into 28b16b8

5045044

modular-magician added the downstream-generated label Aug 20, 2018

modular-magician merged commit 14bc77e into master Aug 20, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Partial updates on Ansible, Puppet and Chef #373

Partial updates on Ansible, Puppet and Chef #373

rambleraptor commented Jul 27, 2018 •

edited

Loading

modular-magician commented Jul 27, 2018 •

edited by rambleraptor

Loading

modular-magician commented Jul 27, 2018

SirGitsalot commented Jul 27, 2018

rambleraptor commented Jul 27, 2018 •

edited

Loading

rambleraptor commented Jul 27, 2018

danawillow commented Jul 27, 2018

rambleraptor Jul 31, 2018

nat-henderson Aug 7, 2018

nat-henderson Aug 7, 2018

rambleraptor Aug 9, 2018

nat-henderson Aug 10, 2018

modular-magician commented Aug 10, 2018

modular-magician commented Aug 10, 2018 •

edited by rambleraptor

Loading

modular-magician commented Aug 20, 2018

nat-henderson commented Aug 20, 2018

		@@ -0,0 +1,23 @@
		<% update_props = properties_by_custom_update(object.all_user_properties) -%>

Partial updates on Ansible, Puppet and Chef #373

Partial updates on Ansible, Puppet and Chef #373

Conversation

rambleraptor commented Jul 27, 2018 • edited Loading

[all]

[terraform]

[puppet]

[puppet-compute]

[puppet-container]

[puppet-dns]

[puppet-logging]

[puppet-pubsub]

[puppet-resourcemanager]

[puppet-sql]

[puppet-storage]

[chef]

[chef-compute]

[chef-container]

[chef-dns]

[chef-logging]

[chef-sql]

[chef-storage]

[ansible]

modular-magician commented Jul 27, 2018 • edited by rambleraptor Loading

modular-magician commented Jul 27, 2018

SirGitsalot commented Jul 27, 2018

rambleraptor commented Jul 27, 2018 • edited Loading

rambleraptor commented Jul 27, 2018

danawillow commented Jul 27, 2018

rambleraptor Jul 31, 2018

Choose a reason for hiding this comment

nat-henderson Aug 7, 2018

Choose a reason for hiding this comment

nat-henderson Aug 7, 2018

Choose a reason for hiding this comment

rambleraptor Aug 9, 2018

Choose a reason for hiding this comment

nat-henderson Aug 10, 2018

Choose a reason for hiding this comment

modular-magician commented Aug 10, 2018

modular-magician commented Aug 10, 2018 • edited by rambleraptor Loading

modular-magician commented Aug 20, 2018

nat-henderson commented Aug 20, 2018

rambleraptor commented Jul 27, 2018 •

edited

Loading

modular-magician commented Jul 27, 2018 •

edited by rambleraptor

Loading

rambleraptor commented Jul 27, 2018 •

edited

Loading

modular-magician commented Aug 10, 2018 •

edited by rambleraptor

Loading