Add support for MongoDB as a DMS source endpoint

[microamp]

This PR is to add support for MongoDB as a DMS source endpoint.

It re-uses some of the attributes from the top-level namespace (server_name, port, username, password and database_name), but it also introduces MongoDB-specific attributes under mongodb_settings closely mimicking the output of aws dms create-endpoint --generate-cli-skeleton.

{
    ...
    "Username": "",
    "Password": "",
    "ServerName": "",
    "Port": 0,
    "DatabaseName": "",
    ...
    "MongoDbSettings": {
        "Username": "",
        "Password": "",
        "ServerName": "",
        "Port": 0,
        "DatabaseName": "",
        "AuthType": "no",
        "AuthMechanism": "scram_sha_1",
        "NestingLevel": "one",
        "ExtractDocId": "",
        "DocsToInvestigate": "",
        "AuthSource": ""
    }
}

Example usage:

resource "aws_dms_endpoint" "dms_endpoint" {
	...
	endpoint_type = "source"
	engine_name = "mongodb"
	...
	server_name = "..."
	port = ...
	username = "..."
	password = "..."
	database_name = "..."
	...
	mongodb_settings {
		auth_type = "PASSWORD"
		auth_mechanism = "DEFAULT"
		nesting_level = "NONE"
		extract_doc_id = "false"
		docs_to_investigate = "1000"
		auth_source = "admin"
	}
}

Here's an acceptance test for the new endpoint support.

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSDmsEndpointMongoDb'

==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSDmsEndpointMongoDb -timeout 120m
=== RUN   TestAccAWSDmsEndpointMongoDb
--- PASS: TestAccAWSDmsEndpointMongoDb (60.58s)
PASS
ok     github.com/microamp/terraform-provider-aws/aws 60.596s

Thanks. Please let me know if I missed anything. Still very new to Terraform. :)

[bflad]

Hi @microamp 👋 Thanks for submitting this! Can you please see the below and let us know if you have any questions?

[bflad]

To set TypeList or TypeSet nested attributes into the Terraform state, we need to create a function that outputs a slice. These current d.Set() calls are likely returning an error that is not caught and silently ignoring the issue, which means that Terraform is going to "pass through" the Terraform configuration as-is into the state and it will never get updated from the API.

Here's an example implementation:

// Always perform error checking when setting non-scalar (TypeList, TypeSet, TypeMap) attributes
if err := d.Set("mongo_settings", flattenDmsMongoDbSettings(endpoint.MongoDbSettings)); err != nil {
  return fmt.Errorf("error setting mongo_settings: %s", err)
}

// Outside the read function
func flattenDmsMongoDbSettings(settings *dms.MongoDbSettings) []map[string]interface{} {
  if settings == nil {
    return []map[string]interface{}{}
  }

  m := map[string]interface{}{
    "auth_mechanism": aws.StringValue(settings.AuthMechanism),
    "auth_source": aws.StringValue(settings.AuthSource),
    "docs_to_investigate": aws.StringValue(settings.DocsToInvestigate),
    "extract_doc_id": aws.StringValue(settings.ExtractDocId),
    "nesting_level": aws.StringValue(settings.NestingLevel),
  }

  return []map[string]interface{}{m}
}

[bflad]

We should move these to the Default parameter in the resource schema: https://www.terraform.io/docs/extend/schemas/schema-behaviors.html#default

e.g.

"auth_type": {
  Type:     schema.TypeString,
  Optional: true,
  Default: "PASSWORD",
},

[bflad]

This attribute is missing documentation. 😄

We may start seeing differences that are something like this if the argument is not present in the Terraform configuration (regardless of whether or not all the children attributes are the default values):

mongodb_settings.#: "1" => "0"

Due to some limitations with Terraform's helper/schema, we will likely need to workaround this with a DiffSuppressFunc on the mongodb_settings attribute:

DiffSuppressFunc: func(k, old, new string, d *schema.ResourceData) bool {
  if old == "1" && new == "0" {
    return true
  }
  return false
},

[microamp]

Hi @bflad 👋

Thanks a lot for the comments, super helpful! I've included the suggested changes then squashed the commits afterwards. Please let me know if everything is in its place now.

[bflad]

Looks good @microamp! 🚀

3 tests passed (all tests)
=== RUN   TestAccAWSDmsEndpointBasic
--- PASS: TestAccAWSDmsEndpointBasic (11.39s)
=== RUN   TestAccAWSDmsEndpointMongoDb
--- PASS: TestAccAWSDmsEndpointMongoDb (11.68s)
=== RUN   TestAccAWSDmsEndpointDynamoDb
--- PASS: TestAccAWSDmsEndpointDynamoDb (29.84s)

[bflad]

This has been released in version 1.17.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!