[Meta] Handle settings drift

[mattwynne]

Any of the settings being handled by these scripts could be changed manually. How do we handle when the actual settings have drifted away from what's declared here in this repo?

Scenarios:

1. An existing repo already has Maintain access for the core team. We bring it under management of these scripts, but that's not defined in the config scripts. Will that access be removed when the config is applied?
2. ...

[mattwynne]

It looks like this may not be entirely possible: pulumi/pulumi-github#160

I think we might be able to do something with import that diffs against existing imported files, as long as we don't touch them manually.

[mattwynne]

It looks like this may not be entirely possible: pulumi/pulumi-github#160

I'm attempting to do something using the import command and some checked-in "golden master" files, which we can compare against.

[mattwynne]

I've got something just about working.

I had to use a PAT (I used CukeBot's) to be able to fetch the list of teams from GitHub's API to generate the resources list for the import. There's now a job failing as there's been a teeny bit of drift I guess since I last ran the import.

To do:

• refine the job so it show the diff when it fails
• tweak things so it passes
• do some testing and check what's happening with the stack - are we importing from the actual state of github, or just from the stack's cache?

[mattwynne]

It does look like the import command is going to the stack for the state of the settings rather than going to the GitHub API, which is not what we want.

I guess we need to either just use a temporary file-based stack, or create a new stack for each import.

[mattwynne]

I've changed it over to use a file-based stack.

I also learned that the generated ts files don't seem to always be in a consistent order, so checking them isn't going to work.

I'm attempting to use pulumi preview with the --json switch to generate a machine-readable preview of the imported resources, to see if that's better for diffing.

[mattwynne]

It's going to be difficult to use this until pulumi/pulumi-github#168 is fixed.