GRPC routes not working possible bug?

[owenhaynes]

Hi,

Having a play with contour to see if its possible to switch from the nghttpx-ingress controller we currently use for internal load traffic.

This traffic is mostly GRPC traffic over use the basic GRPC server set up, these are insecure connections at the moment.

It looks like the GRPC pass through is not working in contour v0.5.0. The clients make the connections but envoy seems to report 404 not found for the endpoints

[2018-05-03T08:01:04.449Z] "POST /node.HeartBeatService/Heartbeat HTTP/2" 404 NR 0 0 0 - "10.20.2.1" "grpc-go/1.11.3" "2a2652fd-9739-43c9-849b-3448d6d20e34" "status.nodes.internal:80" "-" [2018-05-03T08:01:04.450Z] "POST /node.HeartBeatService/Heartbeat HTTP/2" 404 NR 0 0 0 - "10.20.2.1" "grpc-go/1.11.3" "70da1984-0af0-486c-8211-e14616c377ca" "status.nodes.internal:80" "-" [2018-05-03T08:01:07.747Z] "POST /node.HeartBeatService/Heartbeat HTTP/2" 404 NR 0 0 0 - "10.20.2.1" "grpc-go/1.11.3" "0e7d1cfa-097f-4688-ad03-78da8da7eb24" "status.nodes.internal:80" "-"

The h2 annotations are set on the service
apiVersion: v1 kind: Service metadata: annotations: contour.heptio.com/upstream-protocol.h2: "6001"
am i missing something?

[davecheney]

Can you tell me more about your backend service, when you say "insecure" do you mean they are not using TLS?

[owenhaynes]

Yes by insecure no TLS. Backed service is nothing to complex, just the ordinary set up like GRPC recommends.

Server:

lis, err := net.Listen("tcp", ":6001")
if err != nil {
  //Do some handling
}
s := grpc.NewServer()
//Register services
s.Serve(lis)

Client:

conn, err := grpc.Dial("service:6001", grpc.WithInsecure())
if err != nil {
  //Do some handling
}
defer conn.Close()
//Create clients

[davecheney]

I think what is happening is Envoy expects grpc connections to be using TLS.

[mattalberts]

@davecheney I'm seeing the same issue. I built out a detailed example/setup in #379 to use as reference. The example includes a cert generation and configuring the TLS secret. Requesting a resource via a gRPC client or grpcurl results in a NR (no cluster for URL); however, requesting the resource with curl or a browser triggers a match.

[davecheney]

@owenhaynes thanks for raising this bug, the underlying issue is #210 (and many others) (see linked issue for the underlying envoy issue).

"status.nodes.internal:80"

^ the trailing :80 is causing envoy to not match the route.

I'm going to talk to the upstream envoy devs about a possible workaround.

[davecheney]

Closed by #398