Using InterfaceVPCEndpoint is awkward for custom PrivateLink connections

[mitchlloyd]

• I'm submitting a ...

  • 🪲 bug report
  • 🚀 feature request
  • 📚 construct library gap
  • ☎️ security issue or vulnerability => Please see policy
  • ❓ support request => Please see note at the top of this template.

• What is the current behavior?

Using InterfaceVPCEndpoint seems oriented toward creating endpoints for AWS-managed
services. This makes it awkward to use for connecting to custom VPC Endpoint Services.
Specifically:

1. The privateDNSEnabled: true default doesn't work for custom VPC Endpoint services
   outside of AWS Marketplace. service com.amazonaws.vpce.us-west-2.vpce-svc-abc123 does not provide a private DNS name.

2. Getting the primary DNS name (i.e. the DNS entry that points to other AZs) is challenging.
   Something like this does not work:

this.baseLambdaFunction = new lambda.Function(this, 'Resource', {
    runtime: props.runtime,
    handler: props.handler,
    code: lambda.Code.asset(props.zipPath),
    vpc: props.vpc,
    environment: {
        MY_SERVICE_HOST: vpcEndpoint.vpcDNSEntries[0] // => "#{Token[TOKEN.24]}"
    },
    memorySize: props.memorySize,
});

Instead, getting that value requires several Cfn functions:

const firstEntry = cdk.Fn.select(0, endpoint.attrDnsEntries);
const entryParts = cdk.Fn.split(':', firstEntry);
const primaryDNSName = cdk.Fn.select(1, entryParts);

3. The current service property expects one port which is presumptuous
   because services can listen on many ports.

• What is the expected behavior (or behavior of feature suggested)?

It may be the case that the intended use of InterfaceVPCEndpoint may be different enough
from connecting to custom VPC Endpoint Services to warrant a new class. I don't have any
good names but maybe CustomVPCEndpoint. This new construct would address the 3 issues
noted above.

At the least, it would be nice if InterfaceVPCEndpoint had a primaryDNSName
method so that users don't need to know the exact format of this Cfn value
to get a DNS name.

• What is the motivation / use case for changing the behavior or adding this feature?

Make using custom PrivateLink connections with CDK more convenient.

• Please tell us about your environment:

  • CDK CLI Version: 1.3.0
  • Module Version: 1.3.0
  • OS: OSX Mojave
  • Language: TypeScript

[nzspambot]

@mitchlloyd you can chain the Fn.* together as such:

.dnsName(Fn.select(1, Fn.split(":", Fn.select(0, VpcEndpoint.vpcEndpointDnsEntries))))

And also given that I'm using vpc endpoints to build alias records it might be ideal to also expose the HostedZoneId - this is needed because currently the ARecord builder doesn't support regions/continents so need to use CfnRecordSet as such:

.aliasTarget(CfnRecordSet.AliasTargetProperty.Builder() .dnsName(Fn.select(1, Fn.split(":", Fn.select(0, VpcEndpoint.vpcEndpointDnsEntries)))) .hostedZoneId(Fn.select(0, Fn.split(":", Fn.select(0, VpcEndpoint.vpcEndpointDnsEntries)))) .evaluateTargetHealth(true) .build())

[flemjame-at-amazon]

@mitchlloyd the private DNS portion is fixed as of #5987

[bweigel]

I am wondering if anyone else needs custom domain names for their interface VPC endpoints.
Something that is described in detail in the context of private API Gateways with custom domain names here or here.

Generally one could think of an architecture like this:

Where the endpoint provider provides a VPC endpoint service (like private AWS API Gateway, RabbitMQ for AmazonMQ etc.) and the consumer would want to use a nice custom domain name like <rabbitmq|apigw>.mycompany.local and not the obscure VPC interface endpoint domains (like vpce-<some-id>.vpce-svc-<some-other-id>.eu-central-1.vpce.amazonaws.com).

Typically that would be accomplished via another proxy (NLB) that balances between the ENIs (IPs) of the interface VPC endpoint. The NLB can have a custom domain name, together with a certificate.

Unfortunately (afaik) the ENIs (ergo IPs) of the interface VPCe are only accessible via some API calls (ec2:describeVpcEndpoints to get ENIs of VPCe and ec2:describeNetworkInterfaces to get IPs of ENIs). Currently we solve that via a Custom Resource, but I think it would be nice to have a method or property like [get]PrivateIPs.

What do you think (aws or cdk-team or aws-customers)? Is the above desirable?

[flemjame-at-amazon]

@bweigel private DNS is a feature of CDK now, as of #10780

[bweigel]

@flemjame-at-amazon yes, if you are the provider, you can create a custom domain for VPC endpoint services that are consumed by your customers. In the case of RabbitMQ on AmazonMQ this would be something along the lines of <broker-id>.mq.eu-central-1.amazonaws.com, which AWS creates as a custom domain for that endpoint service. Or at least that is my understanding of it.

However this is not the problem my suggestion would address. I am talking from the customer perspective, looking at the this domainname (<broker-id>.mq.eu-central-1.amazonaws.com). Sure, it might be a custom name, but still with little meaning to me as a customer. As a customer I might need something along the lines of rabbitmq.dev.my-domain.local or rabbitmq.prod.my-domain.local. To make names mean something.

You could argue to just add a CNAME Record that points to <broker-id>.mq.eu-central-1.amazonaws.com, but then the client cannot verify the certificate, bc. it is only valid for <broker-id>.mq.eu-central-1.amazonaws.com.

VPC endpoint service ≠ interface VPC endpoint

Does this make it clearer?

[flemjame-at-amazon]

@bweigel so you want to alias an endpoint to a user-defined name, and hide the certificate of the original service? Would there be any verification of the original service's cert?

[bweigel]

@bweigel so you want to alias an endpoint to a user-defined name, and hide the certificate of the original service?

Only if there is no other way. What I (or rather the people I talk to inside our org) want is a custom domain name (i.e. a name that can be interpreted in our context) for the interface VPC endpoints.

Would there be any verification of the original service's cert?

I guess not, if the SSL termination moved to another place.

[dlants]

I ran into this while trying to set up privatelink for ElasticSearch Cloud

It seems that ElasticSearch did not set up a default dns, so privateDnsEnabled does not work for the com.amazonaws.vpce.us-east-1.vpce-svc-0e42e1e06ed010238 service. Instead you have to create your own private hostedZone with a Cname that maps * to your private endpoint dns.

Naively, I attempted:

    const hostedZone = new route53.PrivateHostedZone(
      this,
      "ElasticSearchHostedZone",
      {
        zoneName: "vpce.us-east-1.aws.elastic-cloud.com",
        vpc: myVpc,
      }
    );

    new route53.CnameRecord(this, "ElasticSearchRecord", {
      zone: hostedZone,
      recordName: "*",
      domainName: endpoint.vpcEndpointDnsEntries[0]
    });

This failed with:

| CREATE_FAILED        | AWS::Route53::RecordSet
[RRSet of type CNAME with DNS name *.vpce.us-east-1.aws.elastic-cloud.com. does not contain exactly one resource record.]

Replacing the CnameRecord declaration with:

    const firstEntry = cdk.Fn.select(0, endpoint.vpcEndpointDnsEntries);
    const entryParts = cdk.Fn.split(':', firstEntry);
    const primaryDNSName = cdk.Fn.select(1, entryParts);

    new route53.CnameRecord(this, "ElasticSearchRecord", {
      zone: hostedZone,
      recordName: "*",
      domainName: primaryDNSName
    });

worked!

[mitchlloyd]

A lot has changed since the original issue post: PrivateLink services can use private dns now, the default for privateDNS was changed. One port on service property doesn't seem to be a problem.

As far as I can tell there is still not a better way to work with the vpcEndpointDnsEntries, but that has enough nuance it should probably be opened as a new issue.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.