Tag published event types in logs and metrics from `PublishGrantEvents` step #489

TylerHendrickson · 2023-11-13T07:59:34Z

Description

This PR adds a type:create/update/delete tag when emitting PublishGrantEvents.event.published metrics, and updates the dashboard to make use of the new tag granularity. In particular, it adds a "sunburst" (pie chart) widget to the Service Summary group, and modifies the existing Stream Item Processing Results widget to show "Published" totals by type.

Additionally, certain log outputs have been updated to include the event type in their structured data.

Note for reviewers: The terraform/datadog_dashboard.tf file on this branch contains changes to widgets unrelated to the ones that make use of the new metric tag. Specifically, the widget_layout.y coordinate offsets have been updated for all top-level widgets (i.e. group containers) – since the new widget takes up 3 additional vertical grid spaces, it increases the height of its group container, which pushes all of the other group widgets down by 3 spaces (i.e. y = old_y + 3).

Testing

Note: We don't generally include assertions for discrete logs or metrics in unit tests; although tests should (still) be passing, they have not been updated.

The Terraform plan posted as a PR comment should confirm that the new and modified widgets are as described above.

Manual tests for Reviewer

Added steps to test feature/functionality manually

Checklist

Provided ticket and description
Provided testing information
Provided adequate test coverage for all new code
Added PR reviewers

github-actions · 2023-11-13T08:05:52Z

Terraform Summary

Step	Result
🖌 Terraform Format & Style	✅
⚙️ Terraform Initialization	✅
🤖 Terraform Validation	✅
📖 Terraform Plan	✅

Output

Validation Output

Success! The configuration is valid.

Plan Output

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # datadog_dashboard.service_dashboard[0] will be updated in-place
  ~ resource "datadog_dashboard" "service_dashboard" {
        id                      = "4jj-dh7-k8z"
        tags                    = []
        # (9 unchanged attributes hidden)

      ~ template_variable {
-           default          = "production" -> null
          ~ defaults         = [
+               "production",
            ]
            name             = "env"
            # (2 unchanged attributes hidden)
        }
      ~ template_variable {
-           default          = "grants-ingest" -> null
          ~ defaults         = [
+               "grants-ingest",
            ]
            name             = "service"
            # (2 unchanged attributes hidden)
        }
      ~ template_variable {
-           default          = "*" -> null
          ~ defaults         = [
+               "*",
            ]
            name             = "version"
            # (2 unchanged attributes hidden)
        }

      ~ widget {
            id = 7060137772964863

          ~ group_definition {
                # (3 unchanged attributes hidden)

              ~ widget {
                    id = 3410808562235972

                  ~ sunburst_definition {
                      ~ title       = "Grant Updates Published in Period" -> "Grant Events Published in Period"
-                       title_align = "left" -> null
-                       title_size  = "16" -> null
                        # (1 unchanged attribute hidden)

+                       legend_inline {
+                           hide_percent = false
+                           hide_value   = false
+                           type         = "inline"
                        }

-                       legend_table {}

                      ~ request {
                          ~ formula {
                              ~ formula_expression = "query1" -> "published_by_type"

-                               limit {
-                                   count = 500 -> null
-                                   order = "desc" -> null
                                }
                            }
                          ~ query {
                              ~ metric_query {
                                  ~ name        = "query1" -> "published_by_type"
                                  ~ query       = "sum:grants_ingest.PublishGrantEvents.event.published{$service, $env}.as_count()" -> "sum:grants_ingest.PublishGrantEvents.event.published{$env,$service} by {type}.as_count()"
                                    # (2 unchanged attributes hidden)
                                }
                            }
-                           style {
-                               palette = "datadog16" -> null
                            }
                        }
                    }

                    # (1 unchanged block hidden)
                }

                # (2 unchanged blocks hidden)
            }

          ~ widget_layout {
              ~ height          = 10 -> 9
                # (4 unchanged attributes hidden)
            }
        }
      ~ widget {
            id = 7370760792383743

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 10 -> 9
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 4221397642482799

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 14 -> 12
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 8477815688600517

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 18 -> 15
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 2104149552975257

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 22 -> 18
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 5046282420920828

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 26 -> 21
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 6907345174500015

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 30 -> 24
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 8614522705467412

          ~ widget_layout {
              ~ height          = 7 -> 6
              ~ y               = 34 -> 27
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 256173609813438

          ~ group_definition {
                # (3 unchanged attributes hidden)

              ~ widget {
                    id = 5838548405759132

                  ~ widget_layout {
                      ~ height          = 16 -> 13
                        # (4 unchanged attributes hidden)
                    }

                    # (1 unchanged block hidden)
                }
              ~ widget {
                    id = 280130678238919

                  ~ timeseries_definition {
                        # (4 unchanged attributes hidden)

                      ~ request {
                            # (2 unchanged attributes hidden)

                          ~ formula {
                              ~ formula_expression = "published" -> "published_by_event_type"
                                # (1 unchanged attribute hidden)

                                # (1 unchanged block hidden)
                            }
                          ~ formula {
                              ~ formula_expression = "total_in_invocation - published - failed" -> "total_in_invocation - published_total - failed"
                                # (1 unchanged attribute hidden)

                                # (1 unchanged block hidden)
                            }

                          ~ query {
                              ~ metric_query {
                                  ~ name        = "published" -> "published_by_event_type"
                                  ~ query       = "sum:grants_ingest.PublishGrantEvents.event.published{$env,$service,$version}.as_count()" -> "sum:grants_ingest.PublishGrantEvents.event.published{$env,$service,$version} by {type}.as_count()"
                                    # (1 unchanged attribute hidden)
                                }
                            }
+                           query {
+                               metric_query {
+                                   data_source = "metrics"
+                                   name        = "published_total"
+                                   query       = "sum:grants_ingest.PublishGrantEvents.event.published{$env,$service,$version}.as_count()"
                                }
                            }

                            # (3 unchanged blocks hidden)
                        }

                        # (1 unchanged block hidden)
                    }

                  ~ widget_layout {
                      ~ height          = 5 -> 4
                        # (4 unchanged attributes hidden)
                    }
                }

                # (6 unchanged blocks hidden)
            }

          ~ widget_layout {
              ~ height          = 17 -> 13
              ~ y               = 41 -> 33
                # (3 unchanged attributes hidden)
            }
        }
      ~ widget {
            id = 4201928207253110

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 58 -> 46
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 934058936296128

          ~ widget_layout {
              ~ height          = 7 -> 6
              ~ y               = 62 -> 49
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
      ~ widget {
            id = 942130410430918

          ~ widget_layout {
              ~ height          = 4 -> 3
              ~ y               = 69 -> 55
                # (3 unchanged attributes hidden)
            }

            # (1 unchanged block hidden)
        }
    }

  # module.DownloadFFISSpreadsheet.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "259c5d1abd973a6d69e82eaee2139fc5-3" -> (known after apply)
      ~ id                     = "87f676bc66e1cd463c45d66d5ac34c2e.zip" -> (known after apply)
      ~ key                    = "87f676bc66e1cd463c45d66d5ac34c2e.zip" -> "9a8232b7a3c33328fb1ea06c88db542b.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "xeY679jQbIkff5KPjSaXfrHPUZxQUFyC" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-DownloadFFISSpreadsheet"
      ~ last_modified                  = "2023-11-13T01:45:07.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadFFISSpreadsheet:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadFFISSpreadsheet:37/invocations" -> (known after apply)
      ~ s3_key                         = "87f676bc66e1cd463c45d66d5ac34c2e.zip" -> "9a8232b7a3c33328fb1ea06c88db542b.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadffisspreadsheet" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadffisspreadsheet"
              ~ "DD_VERSION"                   = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_permission.current_version_triggers["SQSQueueNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "SQSQueueNotification" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "bfedf2c45be284c9df22977803e17bd9-3" -> (known after apply)
      ~ id                     = "123c409c86da87490d561a776b602b26.zip" -> (known after apply)
      ~ key                    = "123c409c86da87490d561a776b602b26.zip" -> "f7faaaf8d0adfa9937a7794349a31061.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "cl.UDvoII.Tir8dr0D5Jt4U92yzwyCZB" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-DownloadGrantsGovDB"
      ~ last_modified                  = "2023-11-13T01:45:07.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadGrantsGovDB:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadGrantsGovDB:37/invocations" -> (known after apply)
      ~ s3_key                         = "123c409c86da87490d561a776b602b26.zip" -> "f7faaaf8d0adfa9937a7794349a31061.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                        = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb"
              ~ "DD_VERSION"                     = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (13 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_permission.current_version_triggers["Schedule"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "Schedule" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "cd2b9514a1472e787ec581dd06780b01-3" -> (known after apply)
      ~ id                     = "ddfc82d52aea92f01d593985ff1c2e08.zip" -> (known after apply)
      ~ key                    = "ddfc82d52aea92f01d593985ff1c2e08.zip" -> "00d5e5305c5e2670ef09d960f4d81480.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "puX3upD75homJW6PUkJml.7fJMpwXIAi" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-EnqueueFFISDownload"
      ~ last_modified                  = "2023-11-13T01:45:07.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-EnqueueFFISDownload:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-EnqueueFFISDownload:37/invocations" -> (known after apply)
      ~ s3_key                         = "ddfc82d52aea92f01d593985ff1c2e08.zip" -> "00d5e5305c5e2670ef09d960f4d81480.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:enqueueffisdownload" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:enqueueffisdownload"
              ~ "DD_VERSION"                   = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "f164209f3ee41744c49b5f6435064f82-3" -> (known after apply)
      ~ id                     = "22727beabcc276c85ec71cfb71993bb6.zip" -> (known after apply)
      ~ key                    = "22727beabcc276c85ec71cfb71993bb6.zip" -> "af95b7d97693f06819e40ab1fae30546.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "5Uls3O3f7Q6v.T4WONDjAKkwdrImx.oO" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-ExtractGrantsGovDBToXML"
      ~ last_modified                  = "2023-11-13T01:45:07.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ExtractGrantsGovDBToXML:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ExtractGrantsGovDBToXML:37/invocations" -> (known after apply)
      ~ s3_key                         = "22727beabcc276c85ec71cfb71993bb6.zip" -> "af95b7d97693f06819e40ab1fae30546.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml"
              ~ "DD_VERSION"                   = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistFFISData.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "edcd46f9b654e0402f1256665e76002a-3" -> (known after apply)
      ~ id                     = "8a12effd1536f9b8848de6ba383d329b.zip" -> (known after apply)
      ~ key                    = "8a12effd1536f9b8848de6ba383d329b.zip" -> "fdd17a7bf6f957a92e9fa6294ad73ba0.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "67s1OXQkR_VmedcjOzb0A2OXOxO7TeTJ" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistFFISData.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PersistFFISData"
      ~ last_modified                  = "2023-11-13T01:45:07.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistFFISData:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistFFISData:37/invocations" -> (known after apply)
      ~ s3_key                         = "8a12effd1536f9b8848de6ba383d329b.zip" -> "fdd17a7bf6f957a92e9fa6294ad73ba0.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                       = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistffisdata" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistffisdata"
              ~ "DD_VERSION"                    = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PersistFFISData.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistGrantsGovXMLDB.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "eea5ca3f48e50316b4328af268c1968c-3" -> (known after apply)
      ~ id                     = "fc7557e40c69ab333d608214acf87716.zip" -> (known after apply)
      ~ key                    = "fc7557e40c69ab333d608214acf87716.zip" -> "14ff5dbc8cee713869c271a8d93ce358.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "7d9ZzQpcAUWqEnK.DpKyXMUQS6G5ezTV" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistGrantsGovXMLDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PersistGrantsGovXMLDB"
      ~ last_modified                  = "2023-11-13T01:45:10.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistGrantsGovXMLDB:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistGrantsGovXMLDB:37/invocations" -> (known after apply)
      ~ s3_key                         = "fc7557e40c69ab333d608214acf87716.zip" -> "14ff5dbc8cee713869c271a8d93ce358.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                       = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistgrantsgovxmldb" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistgrantsgovxmldb"
              ~ "DD_VERSION"                    = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PersistGrantsGovXMLDB.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PublishGrantEvents.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "e073c5f25492b525019f8d5e7b95e5eb-3" -> (known after apply)
      ~ id                     = "c34668e756f10a9bddf8bf4f1ba70473.zip" -> (known after apply)
      ~ key                    = "c34668e756f10a9bddf8bf4f1ba70473.zip" -> "77d703adedc72840a152c7a72d25ec17.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "yezk0Qu8aQXQF1fNz6.BX.AgrkofB294" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PublishGrantEvents.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PublishGrantEvents"
      ~ last_modified                  = "2023-11-13T01:45:10.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PublishGrantEvents:38" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PublishGrantEvents:38/invocations" -> (known after apply)
      ~ s3_key                         = "c34668e756f10a9bddf8bf4f1ba70473.zip" -> "77d703adedc72840a152c7a72d25ec17.zip"
        tags                           = {}
      ~ version                        = "38" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:publishgrantevents" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:publishgrantevents"
              ~ "DD_VERSION"                   = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PublishGrantEvents.module.lambda_function.aws_lambda_permission.current_version_triggers["dynamodb"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "dynamodb" -> (known after apply)
      ~ qualifier           = "38" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ReceiveFFISEmail.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "d199075c04ba8c89b88ea4aa08ac43ed-3" -> (known after apply)
      ~ id                     = "115011292f90ca1f0d172dc49373cce7.zip" -> (known after apply)
      ~ key                    = "115011292f90ca1f0d172dc49373cce7.zip" -> "35596e98ae7c6f8271ee8c7e84da51f9.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = ".Y3PxjjJwJsv664hLYRSSx85qhCAmj.R" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ReceiveFFISEmail.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-ReceiveFFISEmail"
      ~ last_modified                  = "2023-11-13T01:45:07.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ReceiveFFISEmail:36" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ReceiveFFISEmail:36/invocations" -> (known after apply)
      ~ s3_key                         = "115011292f90ca1f0d172dc49373cce7.zip" -> "35596e98ae7c6f8271ee8c7e84da51f9.zip"
        tags                           = {}
      ~ version                        = "36" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                        = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:receiveffisemail" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:receiveffisemail"
              ~ "DD_VERSION"                     = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (12 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ReceiveFFISEmail.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "36" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitFFISSpreadsheet.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "6df8dca45490906e0f3c95682e1eedc4-3" -> (known after apply)
      ~ id                     = "3e9ba9ea945b22cf7978baef9b6a0315.zip" -> (known after apply)
      ~ key                    = "3e9ba9ea945b22cf7978baef9b6a0315.zip" -> "d0636b3ff1b164fb9a6f687aa6ce44b9.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "B.uXy6MTw4aBceH0oLHE6aa7xykwRVEf" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitFFISSpreadsheet.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-SplitFFISSpreadsheet"
      ~ last_modified                  = "2023-11-13T01:45:11.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitFFISSpreadsheet:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitFFISSpreadsheet:37/invocations" -> (known after apply)
      ~ s3_key                         = "3e9ba9ea945b22cf7978baef9b6a0315.zip" -> "d0636b3ff1b164fb9a6f687aa6ce44b9.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                          = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitffisspreadsheet" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitffisspreadsheet"
              ~ "DD_VERSION"                       = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (14 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.SplitFFISSpreadsheet.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitGrantsGovXMLDB.module.lambda_artifact.aws_s3_object.lambda_function must be replaced
-/+ resource "aws_s3_object" "lambda_function" {
+       acl                    = (known after apply)
      ~ bucket_key_enabled     = false -> (known after apply)
+       checksum_crc32         = (known after apply)
+       checksum_crc32c        = (known after apply)
+       checksum_sha1          = (known after apply)
+       checksum_sha256        = (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "485edea0e1fcaf312d398e2ee0aa0a3d-3" -> (known after apply)
      ~ id                     = "32dd20f3998f7d4fbcd9acd85cb121d6.zip" -> (known after apply)
      ~ key                    = "32dd20f3998f7d4fbcd9acd85cb121d6.zip" -> "c8887d7a7affdab43af96c3003db6df3.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ storage_class          = "STANDARD" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "j2qPpiz8JOQAIB7QHQKv8QyVYAXsTmdV" -> (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitGrantsGovXMLDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-SplitGrantsGovXMLDB"
      ~ last_modified                  = "2023-11-13T01:45:07.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitGrantsGovXMLDB:37" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitGrantsGovXMLDB:37/invocations" -> (known after apply)
      ~ s3_key                         = "32dd20f3998f7d4fbcd9acd85cb121d6.zip" -> "c8887d7a7affdab43af96c3003db6df3.zip"
        tags                           = {}
      ~ version                        = "37" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                          = "git.commit.sha:f9825667bd5af760860985b869dafe5ba952b6b7,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitgrantsgovxmldb" -> "git.commit.sha:303fd687acdcd43c7f00303c3cfcc7fc0df72c7d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitgrantsgovxmldb"
              ~ "DD_VERSION"                       = "f9825667bd5af760860985b869dafe5ba952b6b7" -> "303fd687acdcd43c7f00303c3cfcc7fc0df72c7d"
                # (14 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.SplitGrantsGovXMLDB.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "37" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

Plan: 20 to add, 11 to change, 20 to destroy.

Pusher: @as1729, Action: pull_request_target, Workflow: Continuous Integration

TylerHendrickson added 3 commits November 13, 2023 02:27

Tag event.published metrics with modification type

489a59b

Add events-by-type widget to dashboard

66c6c33

Show publishing operation totals by type

2594b12

TylerHendrickson self-assigned this Nov 13, 2023

github-actions bot added enhancement New feature or request go Pull requests that update Go code terraform Pull requests that update Terraform code labels Nov 13, 2023

TylerHendrickson requested a review from a team November 13, 2023 18:07

TylerHendrickson enabled auto-merge (squash) November 14, 2023 00:55

as1729 approved these changes Nov 14, 2023

View reviewed changes

Merge branch 'main' into feat/tag-published-event-type

303fd68

TylerHendrickson merged commit a9973da into main Nov 14, 2023
18 checks passed

TylerHendrickson deleted the feat/tag-published-event-type branch November 14, 2023 15:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tag published event types in logs and metrics from `PublishGrantEvents` step #489

Tag published event types in logs and metrics from `PublishGrantEvents` step #489

TylerHendrickson commented Nov 13, 2023 •

edited

Loading

github-actions bot commented Nov 13, 2023 •

edited

Loading

Tag published event types in logs and metrics from PublishGrantEvents step #489

Tag published event types in logs and metrics from PublishGrantEvents step #489

Conversation

TylerHendrickson commented Nov 13, 2023 • edited Loading

Description

Testing

Manual tests for Reviewer

Checklist

github-actions bot commented Nov 13, 2023 • edited Loading

Terraform Summary

Output

Tag published event types in logs and metrics from `PublishGrantEvents` step #489

Tag published event types in logs and metrics from `PublishGrantEvents` step #489

TylerHendrickson commented Nov 13, 2023 •

edited

Loading

github-actions bot commented Nov 13, 2023 •

edited

Loading