grpc-xds-sample

In this repo:

• Sample Go gRPC application, with code to use xDS features:
  • Client-side configuration
  • Server-side configuration
  • Other gRFCs
• A sample Kubernetes deployment compatible with Istio.

This is intended mostly as a guide for how to make your own gRPC applications compatible with xDS.

As a guide for writing your own applications

Clients

First, your application must import the gRPC xDS package.

grpc-xds-sample/main.go

Lines 14 to 16 in 092b840

	// even if you're not using anything from this import, it should still be included to get the side-effect
	// of installing xDS balancers and resolvers (enabling xds:/// URLs in grpc.Dial)
	"google.golang.org/grpc/xds"

If you don't actually use the import, still include it as a side-effect import like this:

// install xDS resolvers
_ "google.golang.org/grpc/xds"

The next important bit is to make sure your Dial or DialContext calls use the xds:/// scheme. In this app, we build the URL dynamically, but prefix it with xds:///

grpc-xds-sample/main.go

Lines 112 to 117 in 092b840

	return fmt.Sprintf(
	"xds:///%s.%s.svc.cluster.local:%s",
	params["SERVICE_NAME"],
	params["SERVICE_NAMESPACE"],
	params["SERVICE_PORT"],
	)

To enable client-side security, we pass a TransportCredentials option. To allow the control plane to send an empty security configuraiton, we also include a fallback of insecure.

grpc-xds-sample/main.go

Lines 44 to 59 in 092b840

	// tell the gRPC server to let xDS configure security
	clientCreds, err := creds.NewClientCredentials(creds.ClientOptions{
	// allow falling back to insecure if no security configuration is given over xDS
	FallbackCreds: insecure.NewCredentials(),
	})
	if err != nil {
	return nil, err
	}
	conn, err := grpc.Dial(
	// make sure to use the fully-qualified domain name (that's what Istio can resolve)
	// this will look like "xds:///greeter.xdssample.svc.cluster.local:7070"
	xdsURL(),
	// use the credentials defined above
	grpc.WithTransportCredentials(clientCreds),
	)

Servers

For servers, the main step is to create the server with a special constructor:

grpc-xds-sample/main.go

Line 76 in 092b840

server := xds.NewGRPCServer(grpc.Creds(serverCreds))

Note that this returns an xds.GRPCServer. If your protobuf generated Go code used an older version of protoc-gen-go-grpc, it may need to be regenerated so that the RegisterServiceNameServer function accepts this new implementation:

grpc-xds-sample/greeter/foo_grpc.pb.go

Line 65 in 092b840

func RegisterGreeterServer(s grpc.ServiceRegistrar, srv GreeterServer) {

Finally, similarly to clients we use special credentials and a fallback to enable server side security:

grpc-xds-sample/main.go

Lines 68 to 76 in 092b840

	// tell the gRPC server to let xDS configure security
	serverCreds, err := creds.NewServerCredentials(creds.ServerOptions{
	// allow falling back to insecure if no security configuration is given over xDS
	FallbackCreds: insecure.NewCredentials(),
	})
	if err != nil {
	return err
	}
	server := xds.NewGRPCServer(grpc.Creds(serverCreds))

Environment

The code changes alone aren't enough to enable xDS in gRPC. We still need to connect to a control plane. Istio makes this easy. We add an annotation that tells Istio's sidecar injector to do a few things for us:

• Run pilot-agent in a special mode that does not run Envoy proxy. Instead it:
  • Fetches certificates and places them on a volume that the gRPC library can access.
  • Proxies connections from the gRPC library to the istiod control plane and handles authentication.
  • Generates a bootstrap file to tell gRPC how to reach the control plane and where to find data plane certs.
• Set a couple of environment variables to configure the gRPC library:
  • GRPC_XDS_BOOTSTRAP is the path to the bootstrap file the agent generates.
  • GRPC_XDS_EXPERIMENTAL_SECURITY_SUPPORT allows actually utilizing security config for mTLS configured by istiod.

Your application should have the following annotations to work easily with gRPC+xDS:

grpc-xds-sample/deployment.yaml

Lines 27 to 29 in 092b840

	annotations:
	inject.istio.io/templates: grpc-agent
	proxy.istio.io/config: '{"holdApplicationUntilProxyStarts": true}'

The first one sets up the agent and environment variables. The second allows the agent to get everything ready before we try to reach the control plane. If your application is robust to client failures, or to server.Serve failures, this isn't necessary.

Deploying the sample

First, create an injection-enabled namespace:

kubectl create ns xdssample
kubectl label ns xdssample istio-injection=enabled

Then simply deploy the example Service and Deployment:

kubectl -n xdssample apply -f deployment.yaml

The application makes a request to the Service every 5 seconds. We can check the logs to make sure it's working:

kubectl -n xdssample logs $(kubectl -n xdssample get po -ojsonpath='{.items[0].metadata.name}')  

They should look something like this:

2021/08/11 22:57:04 Hello, xDS client! From: greeter-748cc9cbff-zlcfm.
2021/08/11 22:57:09 Hello, xDS client! From: greeter-748cc9cbff-8mgmg.
2021/08/11 22:57:14 Hello, xDS client! From: greeter-748cc9cbff-zlcfm.
2021/08/11 22:57:19 Hello, xDS client! From: greeter-748cc9cbff-8mgmg.
2021/08/11 22:57:24 Hello, xDS client! From: greeter-748cc9cbff-zlcfm.
2021/08/11 22:57:29 Hello, xDS client! From: greeter-748cc9cbff-8mgmg.
2021/08/11 22:57:34 Hello, xDS client! From: greeter-748cc9cbff-zlcfm.