Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make LocalStack work with EndpointResolverV2 #542

Merged
merged 1 commit into from
Dec 12, 2023
Merged

Make LocalStack work with EndpointResolverV2 #542

merged 1 commit into from
Dec 12, 2023

Conversation

TylerHendrickson
Copy link
Member

Ticket #496 #252

Description

This PR implements the last set of fixes necessary to resolve #496. Along with #539, #540, and #541, the following issues are addressed:

  • aws-sdk-go-v2 dependencies that could not be upgraded individually (because they caused test failures) are updated to their latest versions
  • Datadog Lambda extension layer is upgraded to be compatible with the latest AWS Lambda runtime requirements
  • LocalStack overrides for local development can work with AWS endpoint resolver customization

@TylerHendrickson TylerHendrickson self-assigned this Dec 12, 2023
This was linked to issues Dec 12, 2023
[Bug]: Update AWS endpoint resolution #252
Closed
[Bug]: Invocation failures after dependency updates #496
Closed
@github-actions github-actions bot added bug Something isn't working go Pull requests that update Go code labels Dec 12, 2023
@github-actions GitHub Actions
Copy link

Terraform Summary

Step Result
🖌 Terraform Format & Style ✅
⚙️ Terraform Initialization ✅
🤖 Terraform Validation ✅
📖 Terraform Plan ✅

Output

Validation Output 
Success! The configuration is valid.
Plan Output 
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-DownloadFFISSpreadsheet"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadFFISSpreadsheet:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadFFISSpreadsheet:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadffisspreadsheet" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadffisspreadsheet"
              ~ "DD_VERSION"                   = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadFFISSpreadsheet.module.lambda_function.aws_lambda_permission.current_version_triggers["SQSQueueNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "SQSQueueNotification" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (4 unchanged attributes hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-DownloadGrantsGovDB"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadGrantsGovDB:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-DownloadGrantsGovDB:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                        = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:downloadgrantsgovdb"
              ~ "DD_VERSION"                     = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (13 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.DownloadGrantsGovDB.module.lambda_function.aws_lambda_permission.current_version_triggers["Schedule"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "Schedule" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-EnqueueFFISDownload"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-EnqueueFFISDownload:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-EnqueueFFISDownload:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:enqueueffisdownload" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:enqueueffisdownload"
              ~ "DD_VERSION"                   = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.EnqueueFFISDownload.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-ExtractGrantsGovDBToXML"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ExtractGrantsGovDBToXML:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ExtractGrantsGovDBToXML:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:extractgrantsgovdbtoxml"
              ~ "DD_VERSION"                   = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ExtractGrantsGovDBToXML.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistFFISData.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PersistFFISData"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistFFISData:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistFFISData:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                       = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistffisdata" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistffisdata"
              ~ "DD_VERSION"                    = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PersistFFISData.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PersistGrantsGovXMLDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PersistGrantsGovXMLDB"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistGrantsGovXMLDB:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PersistGrantsGovXMLDB:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                       = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistgrantsgovxmldb" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:persistgrantsgovxmldb"
              ~ "DD_VERSION"                    = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PersistGrantsGovXMLDB.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.PublishGrantEvents.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-PublishGrantEvents"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PublishGrantEvents:44" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-PublishGrantEvents:44/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "44" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                      = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:publishgrantevents" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:publishgrantevents"
              ~ "DD_VERSION"                   = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (11 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.PublishGrantEvents.module.lambda_function.aws_lambda_permission.current_version_triggers["dynamodb"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "dynamodb" -> (known after apply)
      ~ qualifier           = "44" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.ReceiveFFISEmail.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-ReceiveFFISEmail"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ReceiveFFISEmail:42" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-ReceiveFFISEmail:42/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "42" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                        = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:receiveffisemail" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:receiveffisemail"
              ~ "DD_VERSION"                     = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (12 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ReceiveFFISEmail.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "42" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitFFISSpreadsheet.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-SplitFFISSpreadsheet"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitFFISSpreadsheet:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitFFISSpreadsheet:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                          = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitffisspreadsheet" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitffisspreadsheet"
              ~ "DD_VERSION"                       = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (14 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.SplitFFISSpreadsheet.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

  # module.SplitGrantsGovXMLDB.module.lambda_function.aws_lambda_function.this[0] will be updated in-place
  ~ resource "aws_lambda_function" "this" {
        id                             = "grants_ingest-SplitGrantsGovXMLDB"
      ~ qualified_arn                  = "arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitGrantsGovXMLDB:43" -> (known after apply)
      ~ qualified_invoke_arn           = "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-2:357150818708:function:grants_ingest-SplitGrantsGovXMLDB:43/invocations" -> (known after apply)
        tags                           = {}
      ~ version                        = "43" -> (known after apply)
        # (21 unchanged attributes hidden)

      ~ environment {
          ~ variables = {
              ~ "DD_TAGS"                          = "git.commit.sha:e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitgrantsgovxmldb" -> "git.commit.sha:09128309bf3279ecff86e60ae66fe3ad477da0ce,git.repository_url:github.com/usdigitalresponse/grants-ingest,handlername:splitgrantsgovxmldb"
              ~ "DD_VERSION"                       = "e7f1cceafe2a66c88bdf0b3089d875bd3acbd5f4" -> "09128309bf3279ecff86e60ae66fe3ad477da0ce"
                # (14 unchanged elements hidden)
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.SplitGrantsGovXMLDB.module.lambda_function.aws_lambda_permission.current_version_triggers["S3BucketNotification"] must be replaced
-/+ resource "aws_lambda_permission" "current_version_triggers" {
      ~ id                  = "S3BucketNotification" -> (known after apply)
      ~ qualifier           = "43" # forces replacement -> (known after apply) # forces replacement
+       statement_id_prefix = (known after apply)
        # (5 unchanged attributes hidden)
    }

Plan: 10 to add, 10 to change, 10 to destroy.

Pusher: @TylerHendrickson, Action: pull_request_target, Workflow: Continuous Integration

@TylerHendrickson TylerHendrickson merged commit 99eec77 into main Dec 12, 2023
18 checks passed
@TylerHendrickson TylerHendrickson deleted the fix/localstack-endpoint-resolution branch December 12, 2023 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@TylerHendrickson TylerHendrickson

Labels
bug Something isn't working go Pull requests that update Go code
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: Invocation failures after dependency updates [Bug]: Update AWS endpoint resolution
1 participant
@TylerHendrickson