google_cloudbuild_trigger Pub/Sub and Webhook configs need repo/branch info

[albrechtflo-hg]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

---

The relatively new feature for creating Pub/Sub based Cloud Build triggers (see #8692 and GoogleCloudPlatform/magic-modules#4931) does not support specifying a source repository and a branch.

While this may be seen as missing feature, please note that the Google docs of the associated gcloud command clearly state that both repo and branch must be specified for a Pub/Sub trigger.

Terraform Version

Terraform v1.0.5
on windows_amd64
+ provider registry.terraform.io/hashicorp/archive v2.2.0
+ provider registry.terraform.io/hashicorp/google v3.81.0
+ provider registry.terraform.io/hashicorp/google-beta v3.81.0
+ provider registry.terraform.io/hashicorp/random v3.1.0

Affected Resource(s)

• google_cloudbuild_trigger

Terraform Configuration Files

resource "google_cloudbuild_trigger" "my_pubsub_build_trigger" {
  provider = google-beta
  name = "pubsub-build-trigger"
  
  pubsub_config {
    topic = google_pubsub_topic.my_pubsub_topic.id
  }

  # No way to specify repo / branch here, as pubsub_config e.g. excludes use of trigger_template.

  filename = "cloudbuild.yaml"
}

Expected Behavior

A working Pub/Sub based Cloud Build trigger is being created.

Actual Behavior

The resulting Cloud Build trigger is useless and does nothing when messages on the Pub/Sub topic arrive (well, because it does not know what to do). Please note that the resulting trigger also has no RUN option in the Google Cloud console, while manually created triggers do.

To be precise, the trigger does trigger a Cloud Build when a Pub/Sub message arrives, but immediately fails with this very nice error message (seen in the Google Cloud Console):

Your build failed to run: INTERNAL

That's all, no more information.

Steps to Reproduce

1. terraform apply

Important Factoids

To me, it seems quite strange that creating such a trigger is allowed at all by the Google REST API. My thought was that this is a bug in the Google API, related to a bug I reported to Google a few months ago, which has been fixed in the meantime: https://issuetracker.google.com/issues/191394513

But in the discussion of the PR, the contributor pointed out that there may be use cases for such triggers (I must admit that I do not fully understand their argument). Personally, I'd stick with the Google docs stating that both repo and branch are required fields.

[melinath]

Looking at the REST API documentation, it seems that all of the pubsub fields available on the API are also available on pubsub_config.

I think you might be looking for the build.source.repo_source field, which allows you to specify a source repository and branch (or tag, or commit sha). It looks like build.source.repo_source is missing an exactly_one_of declaration, so I've created a PR to add that.

If repo_source doesn't do what you need, please give some more details about which parts of the REST API are not supported.

[albrechtflo-hg]

Well, the build.source.repo_source refers to a Build Template. You can use that, but in that case, you have to specify your whole build inside the build trigger, which is very inconvenient if you have complex steps (usually, you would use a filename as specified above to refer to a build script inside the referenced repo). It is impossible to just specify the repo_source there.

So, the current workaround is that you have to specify a complete Build Template within the resource when using Pub/Sub triggers. I am sorry, but this is not the same what the REST API and the gcloud alpha build triggers create pubsub command offers. I'd vote for reopening this bug.

The whole trigger resource looks very confusing to me. Serveral places to define same things etc. I think it needs a complete overhaul. Perhaps you should less refer to the REST API documentation, and more to the gcloud alpha build triggers create * docs, which give a better idea of different allowed combinations (after all, that command also only fires to the REST API, which you can even trace using verbose logging).

[melinath]

@albrechtflo-hg Unfortunately, we are not able to provide a custom experience on top of the API the way that gcloud does. Deviating from the REST API tends to cause more maintenance headaches for us in the long run than it's worth.

If there is a way to accomplish your goal with the REST API without specifying a whole build template, that is definitely something we should support, and it might be considered a bug. However, if it's complicated in the REST API, then that's how it will be in the Terraform provider as well.

[albrechtflo-hg]

@melinath It is really not that complicated:

uri: https://cloudbuild.googleapis.com/v1/projects/<project-id>/locations/global/triggers?alt=json
method: POST
== headers start ==
<snip>
== headers end ==
== body start ==
{"gitFileSource": {"path": "cloudbuild.yaml", "revision": "refs/heads/master", "uri": "my-build-repo"}, 
"name": "test-trigger-with-rest", "pubsubConfig": {"topic": "projects/<project-id>/topics/<topic-name>"}, 
"sourceToBuild": {"ref": "refs/heads/master", "uri": "my-build-repo"}}
== body end ==
==== request end ====
---- response start ----
status: 200

I'd be very happy to see that done with the provider.

[melinath]

Thanks for that sample API request! That clarifies things a lot. It looks like you're asking for two new fields: gitFileSource and sourceToBuild. That makes this an enhancement request rather than a bug, since it's new fields rather than fixing an issue with existing fields.

There's already an enhancement request open for these fields if you'd like to vote for it: #9883

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.