Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Endless Reconcile Loop in RDS Controller Due to Subnet Order Difference in DBSubnetGroup CRD #2083

Open
doryer opened this issue Jun 4, 2024 · 0 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. service/rds Indicates issues or PRs that are related to rds-controller.

Comments

@doryer
Copy link

doryer commented Jun 4, 2024

Describe the bug
Recently, we performed a migration from Terraform-provisioned RDS to RDS Operator using AdoptedResource CRDs. We had multiple DBs with different subnet groups, and the order of the three subnets we were using (across three AZs) was different. We wanted to align them to match the AZ order. Instead of having:

  • Subnet-AZ2

  • Subnet-AZ1

  • Subnet-AZ3
    We aimed to have:

  • Subnet-AZ1

  • Subnet-AZ2

  • Subnet-AZ3
    When we created the adopted resource and then synced the DBSubnetGroup new CRD, the difference was only in the list order, but the subnets were the same. I expected no changes to occur. However, the RDS controller logs indicated that it treated these as six different subnets (three from the first list and three from the second list). Logs:

{ "level": "info", "ts": "2024-06-04T12:47:06.520Z", "logger": "ackrt", "msg": "desired resource state has changed", "kind": "DBSubnetGroup", "namespace": "default", "name": "dummy-sgn", "account": "123456789", "role": "", "region": "eu-west-1", "is_adopted": false, "generation": 1, "diff": [ { "Path": { "Parts": [ "Spec", "SubnetIDs" ] }, "A": [ "subnet-B", "subnet-A", "subnet-C" ], "B": [ "subnet-B", "subnet-A", "subnet-C", "subnet-C", "subnet-B", "subnet-A" ] } ] }

Steps to reproduce

  1. Create a DBSubnetGroup CRD using AdoptedResource based on an existing SubnetGroup.
  2. Apply the same list of subnets, but in a different order.

Expected outcome
The RDS Controller should recognize that these are the same three subnets, so no changes should be performed, avoiding an endless reconcile loop.

Environment

  • Kubernetes version - 1.25
  • Using EKS (yes/no), if so version? Yes, EKS 1.25
  • AWS service targeted (S3, RDS, etc.) - RDS
@a-hilaly a-hilaly added kind/bug Categorizes issue or PR as related to a bug. service/rds Indicates issues or PRs that are related to rds-controller. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. service/rds Indicates issues or PRs that are related to rds-controller.
Projects
None yet
Development

No branches or pull requests

2 participants