feat: Add support to use existing notification channels for alerts #593

ps-occrp · 2024-04-17T15:56:40Z

Add support to use existing notification channels for alerts

modules/backup/variables.tf

imrannayer · 2024-04-17T16:25:05Z

@ps-occrp can u also add a notification channel and pass it to the module in this example so it will become part of the test.

examples/postgresql-backup-provided-service-account/main.tf

imrannayer · 2024-04-18T14:09:44Z

/gcbrun

imrannayer · 2024-04-18T16:50:58Z

@ps-occrp integration test is failing with following error:

estPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: Error: Failed to retrieve project, pid: , err: project: required field is not set
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   with google_monitoring_notification_channel.email,
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   48: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: Error: Failed to retrieve project, pid: , err: project: required field is not set
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185:   58: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-18T14:31:39Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Failed to retrieve project, pid: , err: project: required field is not set

  with google_monitoring_notification_channel.email,
  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
  48: resource "google_monitoring_notification_channel" "email" {


Error: Failed to retrieve project, pid: , err: project: required field is not set

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
        	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: error while running command: exit status 1; 
        	            	Error: Failed to retrieve project, pid: , err: project: required field is not set
        	            	
        	            	  with google_monitoring_notification_channel.email,
        	            	  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
        	            	  48: resource "google_monitoring_notification_channel" "email" {
        	            	
        	            	
        	            	Error: Failed to retrieve project, pid: , err: project: required field is not set
        	            	
        	            	  with module.backup.google_monitoring_notification_channel.email[0],
        	            	  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
        	            	  58: resource "google_monitoring_notification_channel" "email" {
        	            	}
        	Test:       	TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/18 14:31:39 RUN_STAGE env var set to apply

ps-occrp · 2024-04-19T09:58:59Z

@imrannayer Can you run gcbrun again?

imrannayer · 2024-04-19T14:14:36Z

/gcbrun

imrannayer · 2024-04-20T01:08:37Z

@ps-occrp test failed:

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-19T14:36:39Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Failed to retrieve project, pid: , err: project: required field is not set

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
        	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: error while running command: exit status 1; 
        	            	Error: Failed to retrieve project, pid: , err: project: required field is not set
        	            	
        	            	  with module.backup.google_monitoring_notification_channel.email[0],
        	            	  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
        	            	  58: resource "google_monitoring_notification_channel" "email" {
        	            	}
        	Test:       	TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/19 14:36:39 RUN_STAGE env var set to apply
2024/04/19 14:36:39 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (61.60s)
FAIL
FAIL	github.com/terraform-google-modules/terraform-google-sql-db/test/integration/postgresql-backup-provided-service-account	62.358s

ps-occrp · 2024-04-22T21:18:38Z

@imrannayer can you run gcbrun again?

imrannayer · 2024-04-23T01:22:18Z

@ps-occrp failing linit test

imrannayer · 2024-04-23T17:11:06Z

/gcbrun

ps-occrp · 2024-04-24T22:20:31Z

@imrannayer looks like gcbrun failed, can you paste error message?

imrannayer · 2024-04-25T00:04:18Z

@ps-occrp

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:50Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Still creating... [10s elapsed]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Creation complete after 14s [id=projects/ci-sql-db-c35d/locations/us-central1/jobs/sql-backup-example-postgres-e6e8c636]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Details:
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.Help",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "links": [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "description": "Google developers console API activation",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "domain": "googleapis.com",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "metadata": {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "consumer": "projects/972442344775",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "service": "monitoring.googleapis.com"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "reason": "SERVICE_DISABLED"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   with google_monitoring_notification_channel.email,
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   48: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: Details:
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.Help",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "links": [
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "description": "Google developers console API activation",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:         "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "domain": "googleapis.com",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "metadata": {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "consumer": "projects/972442344775",
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:       "service": "monitoring.googleapis.com"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     },
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:     "reason": "SERVICE_DISABLED"
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   }
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: ]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185:   58: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-23T17:36:55Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Google developers console API activation",
        "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
      }
    ]
  },
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/972442344775",
      "service": "monitoring.googleapis.com"
    },
    "reason": "SERVICE_DISABLED"
  }
]

  with google_monitoring_notification_channel.email,
  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
  48: resource "google_monitoring_notification_channel" "email" {


Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Google developers console API activation",
        "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
      }
    ]
  },
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/972442344775",
      "service": "monitoring.googleapis.com"
    },
    "reason": "SERVICE_DISABLED"
  }
]

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
        	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: error while running command: exit status 1; 
        	            	Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
        	            	Details:
        	            	[
        	            	  {
        	            	    "@type": "type.googleapis.com/google.rpc.Help",
        	            	    "links": [
        	            	      {
        	            	        "description": "Google developers console API activation",
        	            	        "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
        	            	      }
        	            	    ]
        	            	  },
        	            	  {
        	            	    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        	            	    "domain": "googleapis.com",
        	            	    "metadata": {
        	            	      "consumer": "projects/972442344775",
        	            	      "service": "monitoring.googleapis.com"
        	            	    },
        	            	    "reason": "SERVICE_DISABLED"
        	            	  }
        	            	]
        	            	
        	            	  with google_monitoring_notification_channel.email,
        	            	  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
        	            	  48: resource "google_monitoring_notification_channel" "email" {
        	            	
        	            	
        	            	Error: Error creating NotificationChannel: googleapi: Error 403: Cloud Monitoring API has not been used in project 972442344775 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
        	            	Details:
        	            	[
        	            	  {
        	            	    "@type": "type.googleapis.com/google.rpc.Help",
        	            	    "links": [
        	            	      {
        	            	        "description": "Google developers console API activation",
        	            	        "url": "https://console.developers.google.com/apis/api/monitoring.googleapis.com/overview?project=972442344775"
        	            	      }
        	            	    ]
        	            	  },
        	            	  {
        	            	    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        	            	    "domain": "googleapis.com",
        	            	    "metadata": {
        	            	      "consumer": "projects/972442344775",
        	            	      "service": "monitoring.googleapis.com"
        	            	    },
        	            	    "reason": "SERVICE_DISABLED"
        	            	  }
        	            	]
        	            	
        	            	  with module.backup.google_monitoring_notification_channel.email[0],
        	            	  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
        	            	  58: resource "google_monitoring_notification_channel" "email" {
        	            	}
        	Test:       	TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/23 17:36:55 RUN_STAGE env var set to apply
2024/04/23 17:36:55 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (302.21s)

ps-occrp · 2024-04-25T08:02:55Z

@imrannayer Example creates a notification channel but looks like monitoring.googleapis.com is not enabled for integration test project. I don't think enabling api should be added to example.

imrannayer · 2024-04-25T12:15:02Z

@imrannayer Example creates a notification channel but looks like monitoring.googleapis.com is not enabled for integration test project. I don't think enabling api should be added to example.

Thats part of project setup. Can u plz add the API?

imrannayer · 2024-04-26T14:32:50Z

/gcbrun

imrannayer · 2024-04-26T17:36:08Z

/gcbrun

imrannayer · 2024-04-26T18:10:08Z

failed with following errors:

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:34Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Still creating... [10s elapsed]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Creation complete after 11s [id=projects/ci-sql-db-6063/locations/us-central1/jobs/sql-backup-example-postgres-0f5349ef]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   with google_monitoring_notification_channel.email,
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   48: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185:   58: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-26T18:02:36Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).

  with google_monitoring_notification_channel.email,
  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
  48: resource "google_monitoring_notification_channel" "email" {


Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
  58: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
        	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:517
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:539
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/utils/stages.go:31
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.13.2/pkg/tft/terraform.go:569
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: error while running command: exit status 1; 
        	            	Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).
        	            	
        	            	  with google_monitoring_notification_channel.email,
        	            	  on main.tf line 48, in resource "google_monitoring_notification_channel" "email":
        	            	  48: resource "google_monitoring_notification_channel" "email" {
        	            	
        	            	
        	            	Error: Error creating NotificationChannel: googleapi: Error 403: Permission denied (or the resource may not exist).
        	            	
        	            	  with module.backup.google_monitoring_notification_channel.email[0],
        	            	  on ../../modules/backup/main.tf line 58, in resource "google_monitoring_notification_channel" "email":
        	            	  58: resource "google_monitoring_notification_channel" "email" {
        	            	}
        	Test:       	TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/26 18:02:36 RUN_STAGE env var set to apply
2024/04/26 18:02:36 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (313.75s)
FAIL

ps-occrp · 2024-04-26T20:45:27Z

@imrannayer this looks like permission issues while creating notification channel. I'm not sure how I can reproduce this or where to look for the issue/fix.

imrannayer · 2024-04-29T14:05:15Z

/gcbrun

imrannayer · 2024-04-29T16:28:46Z

@ps-occrp can u plz add following permission in IAM file

roles/monitoring.editor

ps-occrp · 2024-04-30T10:28:23Z

@imrannayer Thanks, I am not familiar with test suit so I didn't know about this. For future work this will come in handy :)

imrannayer · 2024-04-30T15:52:45Z

/gcbrun

imrannayer · 2024-04-30T16:54:30Z

@ps-occrp

TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: module.backup.google_cloud_scheduler_job.sql_backup[0]: Creation complete after 10s [id=projects/ci-sql-db-25d4/locations/us-central1/jobs/sql-backup-example-postgres-49b11252]
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: Error: Error creating NotificationChannel: googleapi: Error 400: Request was missing field notification_channel.labels[email_address].
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185:   with module.backup.google_monitoring_notification_channel.email[0],
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185:   on ../../modules/backup/main.tf line 66, in resource "google_monitoring_notification_channel" "email":
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185:   66: resource "google_monitoring_notification_channel" "email" {
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z command.go:185: 
TestPostgresqlBackupModuleProvidedServiceAccount 2024-04-30T16:18:26Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; 
Error: Error creating NotificationChannel: googleapi: Error 400: Request was missing field notification_channel.labels[email_address].

  with module.backup.google_monitoring_notification_channel.email[0],
  on ../../modules/backup/main.tf line 66, in resource "google_monitoring_notification_channel" "email":
  66: resource "google_monitoring_notification_channel" "email" {
}
    apply.go:34: 
        	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.13/modules/terraform/apply.go:34
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:518
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:577
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:616
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/utils/stages.go:31
        	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.14.0/pkg/tft/terraform.go:616
        	Error:      	Received unexpected error:
        	            	FatalError{Underlying: error while running command: exit status 1; 
        	            	Error: Error creating NotificationChannel: googleapi: Error 400: Request was missing field notification_channel.labels[email_address].
        	            	
        	            	  with module.backup.google_monitoring_notification_channel.email[0],
        	            	  on ../../modules/backup/main.tf line 66, in resource "google_monitoring_notification_channel" "email":
        	            	  66: resource "google_monitoring_notification_channel" "email" {
        	            	}
        	Test:       	TestPostgresqlBackupModuleProvidedServiceAccount
2024/04/30 16:18:26 RUN_STAGE env var set to apply
2024/04/30 16:18:26 Skipping stage teardown
--- FAIL: TestPostgresqlBackupModuleProvidedServiceAccount (295.13s)
FAIL
FAIL	github.com/terraform-google-modules/terraform-google-sql-db/test/integration/postgresql-backup-provided-service-account	295.999s
FAIL

imrannayer · 2024-04-30T17:57:03Z

@ps-occrp you need to setup create_notification_channel to false. Thats causing integration test to fail.

imrannayer · 2024-05-01T14:39:20Z

/gcbrun

imrannayer · 2024-05-01T14:57:48Z

/gcbrun

feat: Add support to use existing notification channels for alerts

668b89d

ps-occrp requested review from isaurabhuttam, imrannayer and a team as code owners April 17, 2024 15:56

imrannayer reviewed Apr 17, 2024

View reviewed changes

modules/backup/variables.tf Outdated Show resolved Hide resolved

imrannayer reviewed Apr 17, 2024

View reviewed changes

modules/backup/variables.tf Show resolved Hide resolved

Rename vairables and update example

2346d63

ps-occrp commented Apr 18, 2024

View reviewed changes

examples/postgresql-backup-provided-service-account/main.tf Outdated Show resolved Hide resolved

ps-occrp added 2 commits April 18, 2024 16:05

fix: Revert back source in example

9d0aae6

fix: Fix typo

3459f5a

Add missing field for notification channel

9948e92

Add missing project property for google_monitoring_notification_channel

fe1bc7c

fix: Fix linting issues

0499e21

Add monitoring.googleapis.com to activate apis for testing

9a3a154

imrannayer self-assigned this Apr 26, 2024

ps-occrp and others added 2 commits April 26, 2024 23:29

Merge branch 'master' into existing_channels

dc40a4b

Merge branch 'master' into existing_channels

e2dda6c

Add monitoring.editor role to testing account

a79cb3b

Disable email notification creation

28d96ca

imrannayer approved these changes May 1, 2024

View reviewed changes

imrannayer merged commit 1c18d78 into terraform-google-modules:master May 1, 2024
4 checks passed

release-please bot mentioned this pull request May 1, 2024

chore(master): release 20.1.0 #592

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add support to use existing notification channels for alerts #593

feat: Add support to use existing notification channels for alerts #593

ps-occrp commented Apr 17, 2024

imrannayer commented Apr 17, 2024

imrannayer commented Apr 18, 2024

imrannayer commented Apr 18, 2024

ps-occrp commented Apr 19, 2024

imrannayer commented Apr 19, 2024

imrannayer commented Apr 20, 2024

ps-occrp commented Apr 22, 2024

imrannayer commented Apr 23, 2024

imrannayer commented Apr 23, 2024

ps-occrp commented Apr 24, 2024

imrannayer commented Apr 25, 2024

ps-occrp commented Apr 25, 2024

imrannayer commented Apr 25, 2024

imrannayer commented Apr 26, 2024

imrannayer commented Apr 26, 2024

imrannayer commented Apr 26, 2024

ps-occrp commented Apr 26, 2024

imrannayer commented Apr 29, 2024

imrannayer commented Apr 29, 2024

ps-occrp commented Apr 30, 2024

imrannayer commented Apr 30, 2024

imrannayer commented Apr 30, 2024

imrannayer commented Apr 30, 2024

imrannayer commented May 1, 2024

imrannayer commented May 1, 2024

feat: Add support to use existing notification channels for alerts #593

feat: Add support to use existing notification channels for alerts #593

Conversation

ps-occrp commented Apr 17, 2024

imrannayer commented Apr 17, 2024

imrannayer commented Apr 18, 2024

imrannayer commented Apr 18, 2024

ps-occrp commented Apr 19, 2024

imrannayer commented Apr 19, 2024

imrannayer commented Apr 20, 2024

ps-occrp commented Apr 22, 2024

imrannayer commented Apr 23, 2024

imrannayer commented Apr 23, 2024

ps-occrp commented Apr 24, 2024

imrannayer commented Apr 25, 2024

ps-occrp commented Apr 25, 2024

imrannayer commented Apr 25, 2024

imrannayer commented Apr 26, 2024

imrannayer commented Apr 26, 2024

imrannayer commented Apr 26, 2024

ps-occrp commented Apr 26, 2024

imrannayer commented Apr 29, 2024

imrannayer commented Apr 29, 2024

ps-occrp commented Apr 30, 2024

imrannayer commented Apr 30, 2024

imrannayer commented Apr 30, 2024

imrannayer commented Apr 30, 2024

imrannayer commented May 1, 2024

imrannayer commented May 1, 2024