Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: typos in S3 lifecycle rules #272

Merged
merged 3 commits into from
Aug 22, 2023
Merged

Conversation

TylerHendrickson
Copy link
Member

@TylerHendrickson TylerHendrickson commented Aug 19, 2023

Ticket #271

Description

This PR fixes several typos that are currently preventing non-current versions of S3 objects from being properly transitioned to Glacier and/or expired after an expected duration.

Note: When running Terraform CI for this PR, I ran into GitHub's 65535(?)-character limit. I updated the workflow to truncate plans to 65300 characters to work around the issue, although the solution indicated in #66 would be even better, since unnecessary Lambda builds contribute unnecessarily to plan length.

Testing

Note: You can inspect the plan output in the Terraform Summary comment below to verify that the lifecycle rules are being configured correctly.

  1. Deploy these terraform changes (e.g. to a localstack environment)
  2. Verify that the buckets' lifecycle rules include steps to transition objects to Glacier (where configured in terraform/main.tf)
  3. Verify that the buckets' lifecycle rules include steps to delete non-current objects (where configured in terraform/main.tf)

Automated and Unit Tests

  • Added Unit tests

Manual tests for Reviewer

  • Added steps to test feature/functionality manually

Checklist

  • Provided ticket and description
  • Provided testing information
  • Provided adequate test coverage for all new code
  • Added PR reviewers

@TylerHendrickson TylerHendrickson added the bug Something isn't working label Aug 19, 2023
@TylerHendrickson TylerHendrickson self-assigned this Aug 19, 2023
@TylerHendrickson TylerHendrickson requested a review from a team as a code owner August 19, 2023 01:33
@github-actions
Copy link

github-actions bot commented Aug 19, 2023

Terraform Summary

Step Result
🖌 Terraform Format & Style
⚙️ Terraform Initialization
🤖 Terraform Validation
📖 Terraform Plan

Output

Validation Output
Success! The configuration is valid.


Plan Output
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+   create
  ~ update in-place
-/+ destroy and then create replacement
 <= read (data resources)

Terraform will perform the following actions:

  # datadog_metric_metadata.custom["grants_ingest.DownloadGrantsGovDB.source_size"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.DownloadGrantsGovDB.source_size"
-       type            = "gauge" -> null
        # (6 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.event.published"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.event.published"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.grant_data.invalid"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.grant_data.invalid"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.invocation_batch_size"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.invocation_batch_size"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.item_image.build"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.item_image.build"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.item_image.malformatted_field"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.item_image.malformatted_field"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.PublishGrantEvents.record.failed"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.PublishGrantEvents.record.failed"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.created"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.created"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.failed"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.failed"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.skipped"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.skipped"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # datadog_metric_metadata.custom["grants_ingest.SplitGrantsGovXMLDB.opportunity.updated"] will be updated in-place
  ~ resource "datadog_metric_metadata" "custom" {
        id              = "grants_ingest.SplitGrantsGovXMLDB.opportunity.updated"
-       type            = "gauge" -> null
        # (5 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.data.aws_s3_bucket.download_target will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "download_target" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-grantssourcedata-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.DownloadFFISSpreadsheet.data.aws_sqs_queue.ffis_downloads will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_sqs_queue" "ffis_downloads" {
+       arn  = (known after apply)
+       id   = (known after apply)
+       name = "ffis_downloads"
+       tags = (known after apply)
+       url  = (known after apply)
    }

  # module.EnqueueFFISDownload.data.aws_s3_bucket.source_data will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "source_data" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-grantssourcedata-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.EnqueueFFISDownload.data.aws_sqs_queue.ffis_downloads will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_sqs_queue" "ffis_downloads" {
+       arn  = (known after apply)
+       id   = (known after apply)
+       name = "ffis_downloads"
+       tags = (known after apply)
+       url  = (known after apply)
    }

  # module.ExtractGrantsGovDBToXML.data.aws_s3_bucket.source_data will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "source_data" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-grantssourcedata-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.PersistFFISData.data.aws_s3_bucket.prepared_data will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "prepared_data" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-grantsprepareddata-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.ReceiveFFISEmail.data.aws_s3_bucket.email_delivery will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "email_delivery" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-emaildelivery-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.ReceiveFFISEmail.data.aws_s3_bucket.grants_source_data will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "grants_source_data" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-grantssourcedata-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.SplitFFISSpreadsheet.data.aws_s3_bucket.prepared_data will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "prepared_data" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-grantsprepareddata-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.SplitFFISSpreadsheet.data.aws_s3_bucket.source_data will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_s3_bucket" "source_data" {
+       arn                         = (known after apply)
+       bucket                      = "grantsingest-staging-grantssourcedata-357150818708-us-west-2"
+       bucket_domain_name          = (known after apply)
+       bucket_regional_domain_name = (known after apply)
+       hosted_zone_id              = (known after apply)
+       id                          = (known after apply)
+       region                      = (known after apply)
+       website_domain              = (known after apply)
+       website_endpoint            = (known after apply)
    }

  # module.grants_prepared_data_bucket.aws_s3_bucket_lifecycle_configuration.default[0] will be updated in-place
  ~ resource "aws_s3_bucket_lifecycle_configuration" "default" {
        id     = "grantsingest-staging-grantsprepareddata-357150818708-us-west-2"
        # (1 unchanged attribute hidden)

      ~ rule {
            id     = "rule-1"
            # (1 unchanged attribute hidden)

+           noncurrent_version_expiration {
+               noncurrent_days = 2557
            }

+           noncurrent_version_transition {
+               noncurrent_days = 30
+               storage_class   = "GLACIER"
            }

            # (2 unchanged blocks hidden)
        }
    }

  # module.grants_source_data_bucket.aws_s3_bucket_lifecycle_configuration.default[0] will be updated in-place
  ~ resource "aws_s3_bucket_lifecycle_configuration" "default" {
        id     = "grantsingest-staging-grantssourcedata-357150818708-us-west-2"
        # (1 unchanged attribute hidden)

      ~ rule {
            id     = "rule-1"
            # (1 unchanged attribute hidden)

+           noncurrent_version_expiration {
+               noncurrent_days = 2557
            }

+           noncurrent_version_transition {
+               noncurrent_days = 30
+               storage_class   = "GLACIER"
            }

            # (2 unchanged blocks hidden)
        }

        # (1 unchanged block hidden)
    }

  # module.lambda_artifacts_bucket.aws_s3_bucket_lifecycle_configuration.default[0] will be updated in-place
  ~ resource "aws_s3_bucket_lifecycle_configuration" "default" {
        id     = "grantsingest-staging-lambdaartifacts-357150818708-us-west-2"
        # (1 unchanged attribute hidden)

      ~ rule {
            id     = "rule-1"
            # (1 unchanged attribute hidden)

+           noncurrent_version_expiration {
+               noncurrent_days = 90
            }

+           noncurrent_version_transition {
+               noncurrent_days = 30
+               storage_class   = "GLACIER"
            }

            # (2 unchanged blocks hidden)
        }
    }

  # module.DownloadFFISSpreadsheet.module.lambda_execution_policy.data.aws_iam_policy_document.this[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "this" {
+       id                        = (known after apply)
+       json                      = (known after apply)
+       override_policy_documents = []
+       source_policy_documents   = [
+           jsonencode(
                {
+                   Statement = [
+                       {
+                           Action   = "secretsmanager:GetSecretValue"
+                           Effect   = "Allow"
+                           Resource = "arn:aws:secretsmanager:us-west-2:357150818708:secret:grants_ingest-staging-datadog_api_key-JGC7E3"
+                           Sid      = "GetDatadogAPIKeySecretValue"
                        },
                    ]
+                   Version   = "2012-10-17"
                }
            ),
        ]

+       statement {
+           actions   = [
+               "s3:PutObject",
            ]
+           effect    = "Allow"
+           resources = [
+               (known after apply),
            ]
+           sid       = "AllowS3DownloadWrite"
        }
+       statement {
+           actions   = [
+               "sqs:DeleteMessage",
+               "sqs:GetQueueAttributes",
+               "sqs:ReceiveMessage",
            ]
+           effect    = "Allow"
+           resources = [
+               (known after apply),
            ]
+           sid       = "AllowSQSGet"
        }
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
+       account_id = (known after apply)
+       arn        = (known after apply)
+       id         = (known after apply)
+       user_id    = (known after apply)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.data.aws_iam_policy_document.assume_role[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "assume_role" {
+       id   = (known after apply)
+       json = (known after apply)

+       statement {
+           actions = [
+               "sts:AssumeRole",
            ]
+           effect  = "Allow"

+           principals {
+               identifiers = [
+                   "lambda.amazonaws.com",
                ]
+               type        = "Service"
            }
        }
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.data.aws_iam_policy_document.logs[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "logs" {
+       id   = (known after apply)
+       json = (known after apply)

+       statement {
+           actions   = [
+               "logs:CreateLogGroup",
+               "logs:CreateLogStream",
+               "logs:PutLogEvents",
            ]
+           effect    = "Allow"
+           resources = [
+               "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-DownloadFFISSpreadsheet:*",
+               "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-DownloadFFISSpreadsheet:*:*",
            ]
        }
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
+       dns_suffix         = (known after apply)
+       id                 = (known after apply)
+       partition          = (known after apply)
+       reverse_dns_prefix = (known after apply)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
+       description = (known after apply)
+       endpoint    = (known after apply)
+       id          = (known after apply)
+       name        = (known after apply)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.data.external.archive_prepare[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "external" "archive_prepare" {
+       id      = (known after apply)
+       program = [
+           "python3",
+           ".terraform/modules/DownloadFFISSpreadsheet.lambda_function/package.py",
+           "prepare",
        ]
+       query   = {
+           "artifacts_dir"            = "builds"
+           "docker"                   = null
+           "hash_extra"               = ""
+           "hash_extra_paths"         = jsonencode([])
+           "paths"                    = jsonencode(
                {
+                   cwd    = "/home/runner/work/grants-ingest/grants-ingest/terraform"
+                   module = ".terraform/modules/DownloadFFISSpreadsheet.lambda_function"
+                   root   = "."
                }
            )
+           "recreate_missing_package" = "true"
+           "runtime"                  = "provided.al2"
+           "source_path"              = jsonencode(
                [
+                   {
+                       commands = [
+                           "task build-DownloadFFISSpreadsheet",
+                           "cd bin/DownloadFFISSpreadsheet",
+                           ":zip",
                        ]
+                       path     = "./.."
                    },
                ]
            )
        }
+       result  = (known after apply)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_iam_policy.additional_json[0] will be updated in-place
  ~ resource "aws_iam_policy" "additional_json" {
        id        = "arn:aws:iam::357150818708:policy/grants_ingest-staging-DownloadFFISSpreadsheet"
        name      = "grants_ingest-staging-DownloadFFISSpreadsheet"
      ~ policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action   = "secretsmanager:GetSecretValue"
-                       Effect   = "Allow"
-                       Resource = "arn:aws:secretsmanager:us-west-2:357150818708:secret:grants_ingest-staging-datadog_api_key-JGC7E3"
-                       Sid      = "GetDatadogAPIKeySecretValue"
                    },
-                   {
-                       Action   = "s3:PutObject"
-                       Effect   = "Allow"
-                       Resource = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2/sources/*/*/*/ffis.org/download.xlsx"
-                       Sid      = "AllowS3DownloadWrite"
                    },
-                   {
-                       Action   = [
-                           "sqs:ReceiveMessage",
-                           "sqs:GetQueueAttributes",
-                           "sqs:DeleteMessage",
                        ]
-                       Effect   = "Allow"
-                       Resource = "arn:aws:sqs:us-west-2:357150818708:ffis_downloads"
-                       Sid      = "AllowSQSGet"
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags      = {}
        # (4 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_iam_policy.logs[0] will be updated in-place
  ~ resource "aws_iam_policy" "logs" {
        id        = "arn:aws:iam::357150818708:policy/grants_ingest-staging-DownloadFFISSpreadsheet-logs"
        name      = "grants_ingest-staging-DownloadFFISSpreadsheet-logs"
      ~ policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action   = [
-                           "logs:PutLogEvents",
-                           "logs:CreateLogStream",
-                           "logs:CreateLogGroup",
                        ]
-                       Effect   = "Allow"
-                       Resource = [
-                           "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-DownloadFFISSpreadsheet:*:*",
-                           "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-DownloadFFISSpreadsheet:*",
                        ]
-                       Sid      = ""
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags      = {}
        # (4 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_iam_role.lambda[0] will be updated in-place
  ~ resource "aws_iam_role" "lambda" {
      ~ assume_role_policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action    = "sts:AssumeRole"
-                       Effect    = "Allow"
-                       Principal = {
-                           Service = "lambda.amazonaws.com"
                        }
-                       Sid       = ""
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        id                    = "grants_ingest-staging-DownloadFFISSpreadsheet"
        name                  = "grants_ingest-staging-DownloadFFISSpreadsheet"
        tags                  = {}
        # (9 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_event_source_mapping.this["sqs"] must be replaced
-/+ resource "aws_lambda_event_source_mapping" "this" {
-       bisect_batch_on_function_error     = false -> null
      ~ event_source_arn                   = "arn:aws:sqs:us-west-2:357150818708:ffis_downloads" # forces replacement -> (known after apply) # forces replacement
      ~ function_arn                       = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-DownloadFFISSpreadsheet" -> (known after apply)
-       function_response_types            = [] -> null
      ~ id                                 = "dfb153f3-fe8a-4de6-93e4-b850b9a1722a" -> (known after apply)
      ~ last_modified                      = "2023-06-30T21:41:50Z" -> (known after apply)
+       last_processing_result             = (known after apply)
      ~ maximum_record_age_in_seconds      = 0 -> (known after apply)
      ~ maximum_retry_attempts             = 0 -> (known after apply)
      ~ parallelization_factor             = 0 -> (known after apply)
-       queues                             = [] -> null
      ~ state                              = "Enabled" -> (known after apply)
      ~ state_transition_reason            = "USER_INITIATED" -> (known after apply)
-       topics                             = [] -> null
-       tumbling_window_in_seconds         = 0 -> null
      ~ uuid                               = "dfb153f3-fe8a-4de6-93e4-b850b9a1722a" -> (known after apply)
        # (4 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-staging-DownloadFFISSpreadsheet"
      ~ last_modified                  = "2023-08-18T21:17:27.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-DownloadFFISSpreadsheet:19" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-DownloadFFISSpreadsheet:19/invocations" -> (known after apply)
      ~ s3_key                         = "builds/5d55949ea26ea2970cac7d6a706a181f609492e940525b47f5b5d8db343519de.zip" -> (known after apply)
      ~ s3_object_version              = "tZHTj3LnwxpaFfVL2R6KIs7xsbWNZKQN" -> (known after apply)
      ~ source_code_hash               = "hm6fQ7wgXlNEN5mamn5CP2zXSJk9GYXEhksgkIVgmsk=" -> (known after apply)
        tags                           = {}
      ~ version                        = "19" -> (known after apply)
        # (18 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
-               "DD_API_KEY_SECRET_ARN"        = (sensitive value)
-               "DD_APM_ENABLED"               = "true"
-               "DD_CAPTURE_LAMBDA_PAYLOAD"    = "true"
-               "DD_ENV"                       = "staging"
-               "DD_SERVERLESS_APPSEC_ENABLED" = "true"
-               "DD_SERVICE"                   = "grants-ingest"
-               "DD_SITE"                      = "datadoghq.com"
-               "DD_TAGS"                      = "git.commit.sha:74d674ca737d4d90e0820f922b0b7955d5253334,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadffisspreadsheet"
-               "DD_TRACE_ENABLED"             = "true"
-               "DD_VERSION"                   = "74d674ca737d4d90e0820f922b0b7955d5253334"
-               "LOG_LEVEL"                    = "INFO"
-               "TARGET_BUCKET_NAME"           = "grantsingest-staging-grantssourcedata-357150818708-us-west-2"
-               "TZ"                           = "UTC"
            } -> (known after apply)
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_permission.current_version_triggers["SQSQueueNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "SQSQueueNotification" -> (known after apply)
      ~ qualifier           = "19" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_s3_object.lambda_package[0] must be replaced
-/+ resource "aws_s3_object" "lambda_package" {
      ~ bucket_key_enabled     = false -> (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "f07c0b649c531b1503437668e08c053d-2" -> (known after apply)
      ~ id                     = "builds/5d55949ea26ea2970cac7d6a706a181f609492e940525b47f5b5d8db343519de.zip" -> (known after apply)
      ~ key                    = "builds/5d55949ea26ea2970cac7d6a706a181f609492e940525b47f5b5d8db343519de.zip" # forces replacement -> (known after apply) # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ source                 = "builds/5d55949ea26ea2970cac7d6a706a181f609492e940525b47f5b5d8db343519de.zip" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "tZHTj3LnwxpaFfVL2R6KIs7xsbWNZKQN" -> (known after apply)
        # (6 unchanged attributes hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.local_file.archive_plan[0] will be created
+   resource "local_file" "archive_plan" {
+       content              = (known after apply)
+       content_base64sha256 = (known after apply)
+       content_base64sha512 = (known after apply)
+       content_md5          = (known after apply)
+       content_sha1         = (known after apply)
+       content_sha256       = (known after apply)
+       content_sha512       = (known after apply)
+       directory_permission = "0755"
+       file_permission      = "0644"
+       filename             = (known after apply)
+       id                   = (known after apply)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "2090471617008103805" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "filename"  = "builds/5d55949ea26ea2970cac7d6a706a181f609492e940525b47f5b5d8db343519de.zip" -> (known after apply)
          ~ "timestamp" = "1692393413649866200" -> (known after apply)
        }
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.null_resource.sam_metadata_aws_lambda_function[0] must be replaced
-/+ resource "null_resource" "sam_metadata_aws_lambda_function" {
      ~ id       = "2108218780147665521" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "built_output_path"    = "builds/5d55949ea26ea2970cac7d6a706a181f609492e940525b47f5b5d8db343519de.zip" -> (known after apply)
            # (4 unchanged elements hidden)
        }
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-staging-DownloadGrantsGovDB"
      ~ last_modified                  = "2023-08-18T21:17:27.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-DownloadGrantsGovDB:44" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-DownloadGrantsGovDB:44/invocations" -> (known after apply)
      ~ s3_key                         = "builds/a0dfc49f0f62e6d8180225c98326d0c5a2ddabc21c762177425200f4489038f3.zip" -> "builds/eb73b98d52a37ad7e49259f74b9d71ccfbdcd9a0f55d238f3ae0f7fbfe6ecfb5.zip"
      ~ s3_object_version              = "EwgsKiKyz4vlW5YS2AI3lLwDkT5EAhNo" -> (known after apply)
        tags                           = {}
      ~ version                        = "44" -> (known after apply)
        # (19 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                        = "git.commit.sha:74d674ca737d4d90e0820f922b0b7955d5253334,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb" -> "git.commit.sha:7a7d93bccdb70fccaeae02a4dbcbaeaad696616d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb"
              ~ "DD_VERSION"                     = "74d674ca737d4d90e0820f922b0b7955d5253334" -> "7a7d93bccdb70fccaeae02a4dbcbaeaad696616d"
                # (12 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_permission.current_version_triggers["Schedule"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "Schedule" -> (known after apply)
      ~ qualifier           = "44" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_s3_object.lambda_package[0] must be replaced
-/+ resource "aws_s3_object" "lambda_package" {
      ~ bucket_key_enabled     = false -> (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "ae77bd44e5fd11676eb2686b7e62594a-2" -> (known after apply)
      ~ id                     = "builds/a0dfc49f0f62e6d8180225c98326d0c5a2ddabc21c762177425200f4489038f3.zip" -> (known after apply)
      ~ key                    = "builds/a0dfc49f0f62e6d8180225c98326d0c5a2ddabc21c762177425200f4489038f3.zip" -> "builds/eb73b98d52a37ad7e49259f74b9d71ccfbdcd9a0f55d238f3ae0f7fbfe6ecfb5.zip" # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ source                 = "builds/a0dfc49f0f62e6d8180225c98326d0c5a2ddabc21c762177425200f4489038f3.zip" -> "builds/eb73b98d52a37ad7e49259f74b9d71ccfbdcd9a0f55d238f3ae0f7fbfe6ecfb5.zip"
-       tags                   = {} -> null
      ~ version_id             = "EwgsKiKyz4vlW5YS2AI3lLwDkT5EAhNo" -> (known after apply)
        # (6 unchanged attributes hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.local_file.archive_plan[0] will be created
+   resource "local_file" "archive_plan" {
+       content              = jsonencode(
            {
+               artifacts_dir = "builds"
+               build_plan    = [
+                   [
+                       "sh",
+                       "..",
+                       <<-EOT
                            task build-DownloadGrantsGovDB
                            cd bin/DownloadGrantsGovDB
                        EOT,
                    ],
+                   [
+                       "zip:embedded",
+                       "..",
+                       null,
                    ],
                ]
+               filename      = "builds/eb73b98d52a37ad7e49259f74b9d71ccfbdcd9a0f55d238f3ae0f7fbfe6ecfb5.zip"
+               runtime       = "provided.al2"
            }
        )
+       content_base64sha256 = (known after apply)
+       content_base64sha512 = (known after apply)
+       content_md5          = (known after apply)
+       content_sha1         = (known after apply)
+       content_sha256       = (known after apply)
+       content_sha512       = (known after apply)
+       directory_permission = "0755"
+       file_permission      = "0644"
+       filename             = "builds/eb73b98d52a37ad7e49259f74b9d71ccfbdcd9a0f55d238f3ae0f7fbfe6ecfb5.plan.json"
+       id                   = (known after apply)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "2733072013363208382" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "filename"  = "builds/a0dfc49f0f62e6d8180225c98326d0c5a2ddabc21c762177425200f4489038f3.zip" -> "builds/eb73b98d52a37ad7e49259f74b9d71ccfbdcd9a0f55d238f3ae0f7fbfe6ecfb5.zip"
          ~ "timestamp" = "1692393393751661000" -> "1692645590055097200"
        }
    }

  # module.DownloadGrantsGovDB.module.lambda_function.null_resource.sam_metadata_aws_lambda_function[0] must be replaced
-/+ resource "null_resource" "sam_metadata_aws_lambda_function" {
      ~ id       = "5556989245393419975" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "built_output_path"    = "builds/a0dfc49f0f62e6d8180225c98326d0c5a2ddabc21c762177425200f4489038f3.zip" -> "builds/eb73b98d52a37ad7e49259f74b9d71ccfbdcd9a0f55d238f3ae0f7fbfe6ecfb5.zip"
            # (4 unchanged elements hidden)
        }
    }

  # module.EnqueueFFISDownload.module.lambda_execution_policy.data.aws_iam_policy_document.this[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "this" {
+       id                        = (known after apply)
+       json                      = (known after apply)
+       override_policy_documents = []
+       source_policy_documents   = [
+           jsonencode(
                {
+                   Statement = [
+                       {
+                           Action   = "secretsmanager:GetSecretValue"
+                           Effect   = "Allow"
+                           Resource = "arn:aws:secretsmanager:us-west-2:357150818708:secret:grants_ingest-staging-datadog_api_key-JGC7E3"
+                           Sid      = "GetDatadogAPIKeySecretValue"
                        },
                    ]
+                   Version   = "2012-10-17"
                }
            ),
        ]

+       statement {
+           actions   = [
+               "s3:GetObject",
            ]
+           effect    = "Allow"
+           resources = [
+               (known after apply),
            ]
+           sid       = "AllowS3DownloadSourceData"
        }
+       statement {
+           actions   = [
+               "sqs:SendMessage",
            ]
+           effect    = "Allow"
+           resources = [
+               (known after apply),
            ]
+           sid       = "AllowSQSPublish"
        }
    }

  # module.EnqueueFFISDownload.module.lambda_function.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
+       account_id = (known after apply)
+       arn        = (known after apply)
+       id         = (known after apply)
+       user_id    = (known after apply)
    }

  # module.EnqueueFFISDownload.module.lambda_function.data.aws_iam_policy_document.assume_role[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "assume_role" {
+       id   = (known after apply)
+       json = (known after apply)

+       statement {
+           actions = [
+               "sts:AssumeRole",
            ]
+           effect  = "Allow"

+           principals {
+               identifiers = [
+                   "lambda.amazonaws.com",
                ]
+               type        = "Service"
            }
        }
    }

  # module.EnqueueFFISDownload.module.lambda_function.data.aws_iam_policy_document.logs[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "logs" {
+       id   = (known after apply)
+       json = (known after apply)

+       statement {
+           actions   = [
+               "logs:CreateLogGroup",
+               "logs:CreateLogStream",
+               "logs:PutLogEvents",
            ]
+           effect    = "Allow"
+           resources = [
+               "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-EnqueueFFISDownload:*",
+               "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-EnqueueFFISDownload:*:*",
            ]
        }
    }

  # module.EnqueueFFISDownload.module.lambda_function.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
+       dns_suffix         = (known after apply)
+       id                 = (known after apply)
+       partition          = (known after apply)
+       reverse_dns_prefix = (known after apply)
    }

  # module.EnqueueFFISDownload.module.lambda_function.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
+       description = (known after apply)
+       endpoint    = (known after apply)
+       id          = (known after apply)
+       name        = (known after apply)
    }

  # module.EnqueueFFISDownload.module.lambda_function.data.external.archive_prepare[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "external" "archive_prepare" {
+       id      = (known after apply)
+       program = [
+           "python3",
+           ".terraform/modules/EnqueueFFISDownload.lambda_function/package.py",
+           "prepare",
        ]
+       query   = {
+           "artifacts_dir"            = "builds"
+           "docker"                   = null
+           "hash_extra"               = ""
+           "hash_extra_paths"         = jsonencode([])
+           "paths"                    = jsonencode(
                {
+                   cwd    = "/home/runner/work/grants-ingest/grants-ingest/terraform"
+                   module = ".terraform/modules/EnqueueFFISDownload.lambda_function"
+                   root   = "."
                }
            )
+           "recreate_missing_package" = "true"
+           "runtime"                  = "provided.al2"
+           "source_path"              = jsonencode(
                [
+                   {
+                       commands = [
+                           "task build-EnqueueFFISDownload",
+                           "cd bin/EnqueueFFISDownload",
+                           ":zip",
                        ]
+                       path     = "./.."
                    },
                ]
            )
        }
+       result  = (known after apply)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_iam_policy.additional_json[0] will be updated in-place
  ~ resource "aws_iam_policy" "additional_json" {
        id        = "arn:aws:iam::357150818708:policy/grants_ingest-staging-EnqueueFFISDownload"
        name      = "grants_ingest-staging-EnqueueFFISDownload"
      ~ policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action   = "secretsmanager:GetSecretValue"
-                       Effect   = "Allow"
-                       Resource = "arn:aws:secretsmanager:us-west-2:357150818708:secret:grants_ingest-staging-datadog_api_key-JGC7E3"
-                       Sid      = "GetDatadogAPIKeySecretValue"
                    },
-                   {
-                       Action   = "s3:GetObject"
-                       Effect   = "Allow"
-                       Resource = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2/sources/*/*/*/ffis.org/raw.eml"
-                       Sid      = "AllowS3DownloadSourceData"
                    },
-                   {
-                       Action   = "sqs:SendMessage"
-                       Effect   = "Allow"
-                       Resource = "arn:aws:sqs:us-west-2:357150818708:ffis_downloads"
-                       Sid      = "AllowSQSPublish"
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags      = {}
        # (4 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_iam_policy.logs[0] will be updated in-place
  ~ resource "aws_iam_policy" "logs" {
        id        = "arn:aws:iam::357150818708:policy/grants_ingest-staging-EnqueueFFISDownload-logs"
        name      = "grants_ingest-staging-EnqueueFFISDownload-logs"
      ~ policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action   = [
-                           "logs:PutLogEvents",
-                           "logs:CreateLogStream",
-                           "logs:CreateLogGroup",
                        ]
-                       Effect   = "Allow"
-                       Resource = [
-                           "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-EnqueueFFISDownload:*:*",
-                           "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-EnqueueFFISDownload:*",
                        ]
-                       Sid      = ""
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags      = {}
        # (4 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_iam_role.lambda[0] will be updated in-place
  ~ resource "aws_iam_role" "lambda" {
      ~ assume_role_policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action    = "sts:AssumeRole"
-                       Effect    = "Allow"
-                       Principal = {
-                           Service = "lambda.amazonaws.com"
                        }
-                       Sid       = ""
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        id                    = "grants_ingest-staging-EnqueueFFISDownload"
        name                  = "grants_ingest-staging-EnqueueFFISDownload"
        tags                  = {}
        # (9 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-staging-EnqueueFFISDownload"
      ~ last_modified                  = "2023-08-18T21:17:27.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-EnqueueFFISDownload:24" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-EnqueueFFISDownload:24/invocations" -> (known after apply)
      ~ s3_key                         = "builds/4c6072d334f4e70e62cc6afc3495e84193964037bf49734ee9b457bc5e3f135b.zip" -> (known after apply)
      ~ s3_object_version              = "TY3CN3wm0NQpcL5BcwX8QVyGNy1sinAJ" -> (known after apply)
      ~ source_code_hash               = "kiO+0cIqpNHQk1FQUSDjjlR8DoA+qt51NiBxsj/p4Fg=" -> (known after apply)
        tags                           = {}
      ~ version                        = "24" -> (known after apply)
        # (18 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
-               "DD_API_KEY_SECRET_ARN"        = (sensitive value)
-               "DD_APM_ENABLED"               = "true"
-               "DD_CAPTURE_LAMBDA_PAYLOAD"    = "true"
-               "DD_ENV"                       = "staging"
-               "DD_SERVERLESS_APPSEC_ENABLED" = "true"
-               "DD_SERVICE"                   = "grants-ingest"
-               "DD_SITE"                      = "datadoghq.com"
-               "DD_TAGS"                      = "git.commit.sha:74d674ca737d4d90e0820f922b0b7955d5253334,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:enqueueffisdownload"
-               "DD_TRACE_ENABLED"             = "true"
-               "DD_VERSION"                   = "74d674ca737d4d90e0820f922b0b7955d5253334"
-               "FFIS_SQS_QUEUE_URL"           = "https://sqs.us-west-2.amazonaws.com/357150818708/ffis_downloads"
-               "LOG_LEVEL"                    = "INFO"
-               "TZ"                           = "UTC"
            } -> (known after apply)
        }

        # (3 unchanged blocks hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "24" # forces replacement -> (known after apply) # forces replacement
      ~ source_arn          = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_permission.unqualified_alias_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "unqualified_alias_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ source_arn          = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_s3_object.lambda_package[0] must be replaced
-/+ resource "aws_s3_object" "lambda_package" {
      ~ bucket_key_enabled     = false -> (known after apply)
      ~ content_type           = "binary/octet-stream" -> (known after apply)
      ~ etag                   = "0e223ee4f0f8cc005e811896f2cab2d6-2" -> (known after apply)
      ~ id                     = "builds/4c6072d334f4e70e62cc6afc3495e84193964037bf49734ee9b457bc5e3f135b.zip" -> (known after apply)
      ~ key                    = "builds/4c6072d334f4e70e62cc6afc3495e84193964037bf49734ee9b457bc5e3f135b.zip" # forces replacement -> (known after apply) # forces replacement
+       kms_key_id             = (known after apply)
-       metadata               = {} -> null
      ~ source                 = "builds/4c6072d334f4e70e62cc6afc3495e84193964037bf49734ee9b457bc5e3f135b.zip" -> (known after apply)
-       tags                   = {} -> null
      ~ version_id             = "TY3CN3wm0NQpcL5BcwX8QVyGNy1sinAJ" -> (known after apply)
        # (6 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.local_file.archive_plan[0] will be created
+   resource "local_file" "archive_plan" {
+       content              = (known after apply)
+       content_base64sha256 = (known after apply)
+       content_base64sha512 = (known after apply)
+       content_md5          = (known after apply)
+       content_sha1         = (known after apply)
+       content_sha256       = (known after apply)
+       content_sha512       = (known after apply)
+       directory_permission = "0755"
+       file_permission      = "0644"
+       filename             = (known after apply)
+       id                   = (known after apply)
    }

  # module.EnqueueFFISDownload.module.lambda_function.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "5581316793654829516" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "filename"  = "builds/4c6072d334f4e70e62cc6afc3495e84193964037bf49734ee9b457bc5e3f135b.zip" -> (known after apply)
          ~ "timestamp" = "1692393413567062000" -> (known after apply)
        }
    }

  # module.EnqueueFFISDownload.module.lambda_function.null_resource.sam_metadata_aws_lambda_function[0] must be replaced
-/+ resource "null_resource" "sam_metadata_aws_lambda_function" {
      ~ id       = "191328334148552412" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "built_output_path"    = "builds/4c6072d334f4e70e62cc6afc3495e84193964037bf49734ee9b457bc5e3f135b.zip" -> (known after apply)
            # (4 unchanged elements hidden)
        }
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_execution_policy.data.aws_iam_policy_document.this[0] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_iam_policy_document" "this" {
+       id                        = (known after apply)
+       json                      = (known after apply)
+       override_policy_documents = []
+       source_policy_documents   = [
+           jsonencode(
                {
+                   Statement = [
+                       {
+                           Action   = "secretsmanager:GetSecretValue"
+                           Effect   = "Allow"
+                           Resource = "arn:aws:secretsmanager:us-west-2:357150818708:secret:grants_ingest-staging-datadog_api_key-JGC7E3"
+                           Sid      = "GetDatadogAPIKeySecretValue"
                        },
                    ]
+                   Version   = "2012-10-17"
                }
            ),
        ]

+       statement {
+           actions   = [
+               "s3:GetObject",
            ]
+           effect    = "Allow"
+           resources = [
+               (known after apply),
            ]
+           sid       = "AllowS3DownloadSourceData"
        }
+       statement {
+           actions   = [
+               "s3:DeleteObject",
+               "s3:GetObject",
+               "s3:GetObjectTagging",
+               "s3:PutObject",
            ]
+           effect    = "Allow"
+           resources = [
+               (known after apply),
            ]
+           sid       = "AllowS3UploadAndMoveTemporaryData"
        }
+       statement {
+           actions   = [
+               "s3:PutObject",
+               "s3:PutObjectTagging",
            ]
+           effect    = "Allow"
+           resources = [
+               (known after apply),
            ]
+           sid       = "AllowS3UploadSourceData"
        }
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.data.aws_caller_identity.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_caller_identity" "current" {
+       account_id = (known after apply)
+       arn        = (known after apply)
+       id         = (known after apply)
+       user_id    = (known after apply)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.data.aws_iam_policy_document.assume_role[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "assume_role" {
+       id   = (known after apply)
+       json = (known after apply)

+       statement {
+           actions = [
+               "sts:AssumeRole",
            ]
+           effect  = "Allow"

+           principals {
+               identifiers = [
+                   "lambda.amazonaws.com",
                ]
+               type        = "Service"
            }
        }
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.data.aws_iam_policy_document.logs[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_iam_policy_document" "logs" {
+       id   = (known after apply)
+       json = (known after apply)

+       statement {
+           actions   = [
+               "logs:CreateLogGroup",
+               "logs:CreateLogStream",
+               "logs:PutLogEvents",
            ]
+           effect    = "Allow"
+           resources = [
+               "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-ExtractGrantsGovDBToXML:*",
+               "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-ExtractGrantsGovDBToXML:*:*",
            ]
        }
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.data.aws_partition.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_partition" "current" {
+       dns_suffix         = (known after apply)
+       id                 = (known after apply)
+       partition          = (known after apply)
+       reverse_dns_prefix = (known after apply)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.data.aws_region.current will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_region" "current" {
+       description = (known after apply)
+       endpoint    = (known after apply)
+       id          = (known after apply)
+       name        = (known after apply)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.data.external.archive_prepare[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "external" "archive_prepare" {
+       id      = (known after apply)
+       program = [
+           "python3",
+           ".terraform/modules/ExtractGrantsGovDBToXML.lambda_function/package.py",
+           "prepare",
        ]
+       query   = {
+           "artifacts_dir"            = "builds"
+           "docker"                   = null
+           "hash_extra"               = ""
+           "hash_extra_paths"         = jsonencode([])
+           "paths"                    = jsonencode(
                {
+                   cwd    = "/home/runner/work/grants-ingest/grants-ingest/terraform"
+                   module = ".terraform/modules/ExtractGrantsGovDBToXML.lambda_function"
+                   root   = "."
                }
            )
+           "recreate_missing_package" = "true"
+           "runtime"                  = "provided.al2"
+           "source_path"              = jsonencode(
                [
+                   {
+                       commands = [
+                           "task build-ExtractGrantsGovDBToXML",
+                           "cd bin/ExtractGrantsGovDBToXML",
+                           ":zip",
                        ]
+                       path     = "./.."
                    },
                ]
            )
        }
+       result  = (known after apply)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_iam_policy.additional_json[0] will be updated in-place
  ~ resource "aws_iam_policy" "additional_json" {
        id        = "arn:aws:iam::357150818708:policy/grants_ingest-staging-ExtractGrantsGovDBToXML"
        name      = "grants_ingest-staging-ExtractGrantsGovDBToXML"
      ~ policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action   = "secretsmanager:GetSecretValue"
-                       Effect   = "Allow"
-                       Resource = "arn:aws:secretsmanager:us-west-2:357150818708:secret:grants_ingest-staging-datadog_api_key-JGC7E3"
-                       Sid      = "GetDatadogAPIKeySecretValue"
                    },
-                   {
-                       Action   = "s3:GetObject"
-                       Effect   = "Allow"
-                       Resource = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2/sources/*/*/*/grants.gov/archive.zip"
-                       Sid      = "AllowS3DownloadSourceData"
                    },
-                   {
-                       Action   = [
-                           "s3:PutObject",
-                           "s3:GetObjectTagging",
-                           "s3:GetObject",
-                           "s3:DeleteObject",
                        ]
-                       Effect   = "Allow"
-                       Resource = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2/tmp/sources/*/*/*/grants.gov/extract.xml"
-                       Sid      = "AllowS3UploadAndMoveTemporaryData"
                    },
-                   {
-                       Action   = [
-                           "s3:PutObjectTagging",
-                           "s3:PutObject",
                        ]
-                       Effect   = "Allow"
-                       Resource = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2/sources/*/*/*/grants.gov/extract.xml"
-                       Sid      = "AllowS3UploadSourceData"
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags      = {}
        # (4 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_iam_policy.logs[0] will be updated in-place
  ~ resource "aws_iam_policy" "logs" {
        id        = "arn:aws:iam::357150818708:policy/grants_ingest-staging-ExtractGrantsGovDBToXML-logs"
        name      = "grants_ingest-staging-ExtractGrantsGovDBToXML-logs"
      ~ policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action   = [
-                           "logs:PutLogEvents",
-                           "logs:CreateLogStream",
-                           "logs:CreateLogGroup",
                        ]
-                       Effect   = "Allow"
-                       Resource = [
-                           "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-ExtractGrantsGovDBToXML:*:*",
-                           "arn:aws:logs:us-west-2:357150818708:log-group:/aws/lambda/grants_ingest-staging-ExtractGrantsGovDBToXML:*",
                        ]
-                       Sid      = ""
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        tags      = {}
        # (4 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_iam_role.lambda[0] will be updated in-place
  ~ resource "aws_iam_role" "lambda" {
      ~ assume_role_policy    = jsonencode(
            {
-               Statement = [
-                   {
-                       Action    = "sts:AssumeRole"
-                       Effect    = "Allow"
-                       Principal = {
-                           Service = "lambda.amazonaws.com"
                        }
-                       Sid       = ""
                    },
                ]
-               Version   = "2012-10-17"
            }
        ) -> (known after apply)
        id                    = "grants_ingest-staging-ExtractGrantsGovDBToXML"
        name                  = "grants_ingest-staging-ExtractGrantsGovDBToXML"
        tags                  = {}
        # (9 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-staging-ExtractGrantsGovDBToXML"
      ~ last_modified                  = "2023-08-18T21:17:27.000+0000" -> (known after apply)
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-ExtractGrantsGovDBToXML:13" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-staging-ExtractGrantsGovDBToXML:13/invocations" -> (known after apply)
      ~ s3_key                         = "builds/c5143a5211eb57d23ada42e67b73abe26e51864d7f7fde39c4ec903b32d8c493.zip" -> (known after apply)
      ~ s3_object_version              = "v7C.trjid3M5Bsrxt6EccBeoIxmuNrqh" -> (known after apply)
      ~ source_code_hash               = "8rMnP0BXdO1m1UJP5u6et2f2DlZlEFMR0ulSE6z+BwY=" -> (known after apply)
        tags                           = {}
      ~ version                        = "13" -> (known after apply)
        # (18 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:74d674ca737d4d90e0820f922b0b7955d5253334,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml" -> "git.commit.sha:7a7d93bccdb70fccaeae02a4dbcbaeaad696616d,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml"
              ~ "DD_VERSION"                   = "74d674ca737d4d90e0820f922b0b7955d5253334" -> "7a7d93bccdb70fccaeae02a4dbcbaeaad696616d"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "13" # forces replacement -> (known after apply) # forces replacement
      ~ source_arn          = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_permission.unqualified_alias_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "unqualified_alias_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ source_arn          = "arn:aws:s3:::grantsingest-staging-grantssourcedata-357150818708-us-west-2" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_s3_object.lambda_package[0] must be replaced
-/+ resource "aws_s3_object" "lambda_package" {
      ~ bucket_key_enabled     = false -> (known after apply)
      ~ content_type 

Pusher: @TylerHendrickson, Action: pull_request, Workflow: Continuous Integration

@TylerHendrickson TylerHendrickson enabled auto-merge (squash) August 19, 2023 01:38
@TylerHendrickson TylerHendrickson merged commit dc7904f into main Aug 22, 2023
@TylerHendrickson TylerHendrickson deleted the fix/271-s3-lifecycle-rules branch August 22, 2023 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Fix incorrectly-configured S3 bucket lifecycle rules for non-current objects
2 participants