ci: Initial Terraform configurations (Work in Progress)

[burkedavison]

See ./.terraform/test.sh for instructions.

These terraform configurations are (currently) designed to create and destroy a new GCP project for each module being tested. To run the script, you must identify a GCP Folder ID in which the project(s) will be created, and a GCP Billing Account ID to assign to the created project(s).

This is a work in progress. We have not decided we want to do this, but it does offer full isolation between integration tests.

terraform and gcloud must be installed. Developed on MacOS. Not yet tested on Linux.

[meltsufin]

Have you looked into how we will run it as part of CI/Kokoro? I guess we'll need to configure Kokoro service account to have permissions on the projects directory.

[burkedavison]

Kokoro is part of the discussion, but not the current focus until we can iron out some of the issues currently being faced. There are some issues with this create+destroy project workflow that I need to resolve; and we haven't made a clear decision yet on whether we want to continue down this path. If we choose to continue having Terraform create+destroy projects, then yes - we'd need to give Kokoro's service account project creation+deletion permissions and define a specific folder for it to use. However, we could also choose to have a single project in which dedicated instances of resources (like database instances, network instances, etc) get created for each test; but this also has issues that would need to be resolved. (For example, we can't delete container networks due to GCE Enforcer firewall rule dependencies, so would that mean our `java-container` ITs litter the project with old networks, or would `java-container` be set up to use only a single network -- or would we have a set of networks that could be allocated+released to allow only a single owner at a time...)

…

On Wed, Sep 21, 2022 at 4:23 PM Mike Eltsufin ***@***.***> wrote: Have you looked into how we will run it as part of CI/Kokoro? I guess we'll need to configure Kokoro service account to have permissions on the products directory. — Reply to this email directly, view it on GitHub <#8436 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJV4PTUCYDEJ7DEWL2HBIUDV7NVCLANCNFSM6AAAAAAQSLS2XE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[suztomo]

I feel these first declaration redundant. The three main.tf has the almost same section. Is there a way to eliminate the redundancy?

[burkedavison]

Investigating

[suztomo]

I think this file can be replaced by a command to find directories that contain Terraform configuration. With that we can eliminate a possibility of having discrepancy (Terraform files are added to modules but forget updating this file). What do you think?

[burkedavison]

Good suggestion. TODO

[suztomo]

What if we don't add generated files. Would this PR becomes much smaller?

[burkedavison]

These three 'template' files aren't actually generated -- they're used to copy and then append to the "actual" generated files which are not in git.

[burkedavison]

See #8555 for continuation of this effort.