feat: add a new GSI to support the API endpoint returning the submission names sorted by creation time

[wmoussa-gc]

Summary | Résumé

While optimizing the call for retrieving new submissions cds-snc/forms-api#33 , we realized we could not sort the submissions by creation time without a performance hit. The current design of the Vault table does not easily allow for this.
This PR adds a new Global Secondary Index (GSI) to include a sort key that would look like "Status#CreatedAt."

[craigzour]

@wmoussa-gc Quickly reviewed the pull request. Not sure if you are about to push it or not but the actual code that insert the new field in DynamoDB is missing. It should be in the saveToVault function.

[bryan-robitaille]

Nice

[craigzour]

Is the all capital letters a constraint from DynamoDB? Or maybe a best practice? If not, I wonder if we should not just use the same format as the one we use for the Status field.

Suggested change

	"Status#CreatedAt": `NEW#${Number(createdAt)}`,
	"Status#CreatedAt": `New#${Number(createdAt)}`,

[wmoussa-gc]

I thought I’d be following the pattern I saw for NAME_OR_CONF, but I also like the idea of sticking with what we used for Status. Perhaps somewhere else in our codebase, we could use ${status}#${Number(createdAt)}.

[github-actions]

⚠ Terrform update available

Terraform: 1.9.5 (using 1.9.2)
Terragrunt: 0.67.2 (using 0.63.2)

[github-actions]

Staging: dynamodb

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 0 to add, 1 to change, 0 to destroy

Show summary

CHANGE	NAME
update	aws_dynamodb_table.vault

Show plan

Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_dynamodb_table.vault will be updated in-place
  ~ resource "aws_dynamodb_table" "vault" {
        id                          = "Vault"
        name                        = "Vault"
        tags                        = {}
        # (13 unchanged attributes hidden)

      + attribute {
          + name = "Status#CreatedAt"
          + type = "S"
        }

      - global_secondary_index {
          - hash_key           = "FormID" -> null
          - name               = "Status" -> null
          - non_key_attributes = [] -> null
          - projection_type    = "ALL" -> null
          - range_key          = "Status" -> null
          - read_capacity      = 0 -> null
          - write_capacity     = 0 -> null
        }
      + global_secondary_index {
          + hash_key           = "FormID"
          + name               = "Status"
          + non_key_attributes = []
          + projection_type    = "ALL"
          + range_key          = "Status"
        }
      + global_secondary_index {
          + hash_key           = "FormID"
          + name               = "StatusCreatedAt"
          + non_key_attributes = [
              + "CreatedAt",
              + "Name",
            ]
          + projection_type    = "INCLUDE"
          + range_key          = "Status#CreatedAt"
        }

        # (6 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"

Show Conftest results

WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.audit_logs"]
WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.reliability_queue"]
WARN - plan.json - main - Missing Common Tags: ["aws_dynamodb_table.vault"]

22 tests, 19 passed, 3 warnings, 0 failures, 0 exceptions