S3 config changes to use S3 FIPS endpoints on FIPS hosts

[harishxr]

Summary

This pull request enables ECS Agent on linux to make S3 calls using S3 FIPS Endpoints when FIPS mode is enabled on the host machine.

Implementation details

• Updated the NewS3ManagerClient and NewS3Client functions to use cfg.UseFIPSEndpoint to call FIPS S3 endpoints when FIPS mode is detected on the host.
• Modified the getRegionFromBucket function to use s3manager.GetBucketRegionWithClient with options to ensure virtual-hosted-style URLs are used when FIPS mode is enabled on the host.
• Added logging statements to indicate when FIPS compliant S3 endpoints and virtual-hosted-style URLs are being used.

Testing

• Built test agent using the changes made in the PR and ran agent on FIPS Enabled host.
• Agent was able to identify it running in a FIPS enabled env based on the following log

/var/log/ecs/ecs-agent.log.2024-06-10-18:level=info time=2024-06-10T18:33:19Z msg="FIPS mode detected on the host"

• Created an S3 bucket and placed a test1.env file in the S3 bucket with the following content

TEST1=ABC
TEST2=XYZ

• Ran a task with the following task definition

{
    "taskDefinitionArn": "arn:aws:ecs:us-west-2:redacted:task-definition/fips-test:2",
    "containerDefinitions": [
        {
            "name": "sleepy300",
            "image": "busybox",
            "cpu": 100,
            "memory": 10,
            "portMappings": [],
            "essential": true,
            "command": [
                "sleep",
                "10000000"
            ],
            "environment": [],
            "environmentFiles": [
                {
                    "value": "arn:aws:s3:::redacted/test1.env",
                    "type": "s3"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],
            "systemControls": []
        }
    ],
    "family": "fips-test",
    "taskRoleArn": "arn:aws:iam::redacted:role/TaskIAMRoleArn",
    "executionRoleArn": "arn:aws:iam::redacted:role/TaskExecutionRoleARN",
    "networkMode": "host",
    "revision": 2,
    "volumes": [],
    "status": "ACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role-network-host"
        },
        {
            "name": "ecs.capability.env-files.s3"
        },
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EXTERNAL",
        "EC2"
    ],
    "cpu": "256",
    "memory": "512",
    "registeredAt": "2024-05-28T22:54:48.812Z",
    "registeredBy": "arn:aws:sts::redacted:assumed-role/Admin/test",
    "tags": []
}

• Verified in the ECS console for the running task under Environment Variables and Files that the env file was attached to the container.
• Logged onto the running container using docker exec -it <container_id> sh and executed the following commands to access the values defined in the env files

/ # echo $TEST1
ABC
/ # echo $TEST2
XYZ

• Verified using the logs that FIPS S3 Endpoints are being used

/var/log/ecs/ecs-agent.log.2024-06-10-18:level=debug time=2024-06-10T18:35:24Z msg="FIPS mode detected, using FIPS-compliant S3 endpoint"
/var/log/ecs/ecs-agent.log.2024-06-10-18:level=debug time=2024-06-10T18:35:24Z msg="FIPS mode detected, using virtual-host–style URLs for bucket location"

New tests cover the changes:
Yes

Description for the changelog

S3 config changes to use S3 FIPS endpoints on FIPS hosts
Does this PR include breaking model changes? If so, Have you added transformation functions?

No

References

• virtual-hosted–style URL - https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
• GetBucketRegionWithClient function - https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#GetBucketRegionWithClient
• s3manager.GetBucketRegionWithClient will cover the special region case EU and us-east-1 per https://github.com/aws/aws-sdk-go/blob/main/service/s3/bucket_location.go#L27

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.