Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: InvalidFleetConfig The fleet configuration contains duplicate instance pools. #2904

Closed
dgokcin opened this issue Jan 21, 2023 · 8 comments · Fixed by #3039
Closed

Comments

@dgokcin
Copy link

dgokcin commented Jan 21, 2023

Helloo

Trying to deploy the module to my infrastructure and having some problems with the scale-up lambda.

Any idea why I might have an invalid fleet config? Thanks!

include {
  path = find_in_parent_folders()
}

locals {
  common_vars = yamldecode(file(find_in_parent_folders("common_vars.yaml")))
}

terraform {
  source = "../../..//modules/terraform-aws-github-runner"
}

inputs = {
  aws_region            = "eu-west-1"
  vpc_id                = dependency.vpc.outputs.vpc_id
  subnet_ids            = dependency.vpc.outputs.private_subnets
  lambda_s3_bucket      = "myorg-pg-gh-actions-lambdas"
  runners_lambda_s3_key = "runners.zip"
  syncer_lambda_s3_key  = "runner-binaries-syncer.zip"
  webhook_lambda_s3_key = "webhook.zip"

  prefix = "gh-ci"
  github_app = {
    key_base64     = "key"
    id             = "id"
    webhook_secret = "webhook_secret"
  }

  create_service_linked_role_spot = true

  enable_organization_runners = true
  runner_extra_labels         = "default,example"

  instance_types = ["m5.large", "c5.large"]

  # override delay of events in seconds
  delay_webhook_event   = 5
  runners_maximum_count = 1

  # set up a fifo queue to remain order
  enable_fifo_build_queue = true

  # override scaling down
  scale_down_schedule_expression = "cron(* * * * ? *)"
  # enable this flag to publish webhook events to workflow job queue
  # enable_workflow_job_events_queue  = true

  enable_user_data_debug_logging_runner = true

  tags = local.common_vars.tags
}

dependency "vpc" {
  config_path = "../vpc"
}

image

@dgokcin
Copy link
Author

dgokcin commented Jan 22, 2023

update: this was the default configuration. I also tried deploying the module with multiple_runners. I believe the following code is failing and I have no idea what is going on. This is my first time working with spot instances so I really appreciate any help.

Thanks

    let fleet;
    try {
        // see for spec https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html
        fleet = await ec2
            .createFleet({
            LaunchTemplateConfigs: [
                {
                    LaunchTemplateSpecification: {
                        LaunchTemplateName: runnerParameters.launchTemplateName,
                        Version: '$Default',
                    },
                    Overrides: generateFleetOverrides(runnerParameters.subnets, runnerParameters.ec2instanceCriteria.instanceTypes, amiIdOverride),
                },
            ],
            SpotOptions: {
                MaxTotalPrice: runnerParameters.ec2instanceCriteria.maxSpotPrice,
                AllocationStrategy: runnerParameters.ec2instanceCriteria.instanceAllocationStrategy,
            },
            TargetCapacitySpecification: {
                TotalTargetCapacity: numberOfRunners,
                DefaultTargetCapacityType: runnerParameters.ec2instanceCriteria.targetCapacityType,
            },
            TagSpecifications: [
                {
                    ResourceType: 'instance',
                    Tags: [
                        { Key: 'ghr:Application', Value: 'github-action-runner' },
                        { Key: 'Type', Value: runnerParameters.runnerType },
                        { Key: 'Owner', Value: runnerParameters.runnerOwner },
                    ],
                },
            ],
            Type: 'instant',
        })
            .promise();
    }
    catch (e) {
        logger.warn('Create fleet request failed.', e);
        throw e;
    }

@dgokcin dgokcin changed the title InvalidFleetConfig The fleet configuration contains duplicate instance pools. Error: InvalidFleetConfig The fleet configuration contains duplicate instance pools. Jan 22, 2023
@jpalomaki
Copy link
Contributor

@dgokcin Do your private subnets (which you seem to inject via a terragrunt dependency) reside in different availability zones?

(Quick google search returns an issue in an unrelated repo, but exhibiting the same error message)

@dgokcin
Copy link
Author

dgokcin commented Jan 23, 2023

yeah I figured it out from that exact post. in my test environment, i have multiple subnets in the same az. just providing a single private subnet resolved my problem.

thanks!

@jpalomaki
Copy link
Contributor

jpalomaki commented Jan 24, 2023

@npalm Maybe it'd be a good idea to highlight this requirement (configured subnets must reside in different AZs) in the subnet_ids input documentation?

@dgokcin
Copy link
Author

dgokcin commented Jan 24, 2023

@npalm Maybe it'd be a good idea to highlight this requirement (configured subnets must reside in different AZs) in the subnet_ids input documentation?

I agree since I was able to figure it out by chance after several hours of reaearching on a different repo.

@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the Stale label Feb 24, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 6, 2023
@IAXES
Copy link

IAXES commented Mar 25, 2024

Just a heads-up: this can also occur (false/misleading positive) if one accidentally enters duplicate EC2 instance-types in the list-of-string supplied to this module. Lesson learned: wrapped it with the distinct() function.

@garbelini
Copy link

@IAXES You saved me what could have been hours of frustration!! ❤️ ❤️ ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants