aws_security_group_rule InvalidPermission.Duplicate - Should be additive

[ghost]

This issue was originally opened by @chrisferry as hashicorp/terraform#1874. It was migrated here as a result of the provider split. The original body of the issue is below.

---

I have a module which creates an ASG tied to an ELB
Within that module there is:

resource "aws_security_group_rule" "allow_elb_http_in" {
  type                     = "ingress"
  from_port                = "${var.elb_listen_port}"
  to_port                  = "${var.elb_listen_port}"
  protocol                 = "tcp"
  security_group_id        = "${var.security_group}"
  source_security_group_id = "${var.elb_security_group}"
}

I have several ASGs in the same APP-SG and several ELBs in the same ELB-SG
Some have overlapping listen ports which causes the following error when I try to apply:

* Error authorizing security group rules rules: InvalidPermission.Duplicate: the specified rule "peer: sg-xxxxxxxx, TCP, from port: 80, to port: 80, ALLOW" already exists

I think that the resource should allow for additive rules due to how the modules work

[justinretzolk]

Hey @chrisferry 👋 Thank you for taking the time to file this issue. Given that there's been quite a few AWS provider releases since you initially filed it, can you confirm whether you're still experiencing this behavior?

[ryan-omni3]

Yes. Just tested on version 3.63.0. This is still an issue.

[justinretzolk]

Hey @ryan-omni3 👋 Thank you very much for the update! By chance, can you provide the Terraform configuration so that we can attempt to reproduce and look into this more as time allows?

[joelleigh]

I get this error as well. When I rerun the same apply job though, it goes through. This is breaking some of our pipelines, would like to see this run successfully on the first run.

[justinretzolk]

Hey @joelleigh 👋 Thank you for the additional report! So that we have enough information to try to look into this, can you supply the version of Terraform and the version of the AWS Provider that you're running, along with a sample configuration that exhibits this behavior?

[joelleigh]

@justinretzolk
Terraform v1.1.7
on windows_amd64

• provider registry.terraform.io/hashicorp/aws v4.0.0

Your version of Terraform is out of date! The latest version
is 1.1.8. You can update by downloading from https://www.terraform.io/downloads.html

We have multiple beanstalk environments, each needing to add to the default vpc security group to allow ingress

resource "aws_security_group" "eb-security-group-game" {

  name   = "${var.service_name}-${var.service_environment}-eb-game"
  vpc_id = data.aws_vpc.default.id
  tags   = {
    Name        = "${var.service_name}-${var.service_environment}"
    Service     = var.service_name
    Environment = var.service_environment
  }
}

// - Allows incoming traffic on port 80
resource "aws_security_group_rule" "eb-ingress-80" {
  type                     = "ingress"
  from_port                = 80
  to_port                  = 80
  protocol                 = "tcp"
  security_group_id        = aws_security_group.eb-security-group-game.id
  source_security_group_id = aws_default_security_group.default.id
}

[justinretzolk]

Hey @joelleigh -- thanks for providing that additional info! With that in mind, I used the following sample configuration to attempt to reproduce this. Unfortunately, I've so far not been able to reproduce the errors you saw, using Terraform 1.1.8 and AWS Provider 4.0.0.

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_security_group" "main" {
  name   = "jretzolk-test-sg-main"
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group" "secondary" {
  count  = 60 # the max number of ingress rules allowed for a given SG
  name   = "jretzolk-test-sg-secondary-${count.index}"
  vpc_id = aws_vpc.main.id
}

# Note: Because of the `for_each` here, this resource must be either commented out and then uncommented
# on a second apply, or a targeted apply must be run against `aws_security_group.secondary`, followed by a
# subsequent apply.
resource "aws_security_group_rule" "main" {
  for_each                 = toset(aws_security_group.secondary.*.id)
  type                     = "ingress"
  from_port                = 80
  to_port                  = 80
  protocol                 = "tcp"
  security_group_id        = aws_security_group.main.id
  source_security_group_id = each.key
}

You mentioned that it seems to work on a second apply, so I recognize this is intermittent and may be difficult to catch, but does it happen consistently enough that it would be possible to reproduce the issue with debug logging turned on so that we can take a look at those debug logs?

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!