Reflection issues

[jamisonhyatt]

I have a proto set with 3 gRPC services (and some external dependencies) that compiles appropriately on the protobuf side, compiles and runs in go, and clients work when interacting with the server, but reflection fails within grpcurl on the 2 services which have dependencies in other packages.

I recreated a fake weatherman repo with a similar setup to provide an example.

proto & generated code repo: https://github.com/jamisonhyatt/grpc-multi-pkg-protos
Client gist: https://gist.github.com/jamisonhyatt/dd9d93b978472c960c9fa81b9f39fcf9

Here's tree of the protoset

└── protos
    ├── desktop_svc
    │   └── desktop_svc.proto
    ├── external
    │   ├── location
    │   │   └── location.proto
    │   └── weather
    │       ├── hurricanes.proto
    │       └── weather.proto
    ├── mobile_svc
    │   └── mobile_svc.proto
    └── weatherman_svc.proto

list returns

desktop.Desktop
grpc.reflection.v1alpha.ServerReflection
mobile.Mobile
weatherman.Weatherman

weatherman.Weatherman.Healthcheck returns

{
  "healthy": true
}

list mobile.Mobile throws an error

Failed to list methods for service "mobile.Mobile": Symbol not found: mobile.Mobile
caused by: File not found: location/location.proto

I'd like to avoid refactoring the packages (which does work, but I have to flatten them) but I'd like to know of this is a symptom of how I have the packae imports, the gRPC reflection service, or the reflection in grpcurl itself.

My feeling though, is that if the protos compile, the server compiles, the reflection service should have enough information to define the types. Any feedback is appreciated.

Edit: just for posterity, building the protosets and using those with grpcurl does work.

[jhump]

@jamisonhyatt, this is a problem in the Go protobuf runtime regarding how file descriptors that are compiled into your binary are "linked".

They are linked purely by name. What that means is that the name (and relative path) used to compile a proto with protoc must exactly match how all other files will import it.

In your test repo, note that location.proto was compiled with a command that simply referenced that file, with no relative path: https://github.com/jamisonhyatt/grpc-multi-pkg-protos/blob/master/pkg/external/location/location.pb.go#L227. This is how it gets registered so that reflective operations can find it.

However it is imported with a location/ relative path prefix. Since location/location.proto != location.proto the reflection operations that attempt to gather all necessary file descriptors fail.

You can fix this by adjusting the include path arguments to protoc so that you import it as simply location.proto OR by adjusting the invocation when compiling this file so that you reference it as location/location.proto when compiling it.

[jamisonhyatt]

This was a phenomenal explanation, and helped me out a lot. I appreciate the details, and the raising of the external issue, as well.

I'm totally cool with closing this issue.

[antoniomo]

I came here with exactly the same issue, perfect explanation :)

[freb]

I just ran into this, except I have no caused by: output.

If you're in the same boat as me, it is probably because you are using gogoprotobuf with protoc-gen-gogo*. The generated code has known issues with reflection.

See: https://jbrandhorst.com/post/gogoproto/#reflection

[tanjunchen]

see https://github.com/tanjunchen/grpc-test-demo

My project can run, and the go client can call the go server grpc service.
go client will return prod_stock:211 status:{code:"200" msg:"success"} when it calls the go grpc service.
However, when the following command is executed, the service will report an error, and I want to know the reason.
I guess that it maybe the reason of grpc/grpc-go#1873.
@jhump

D:\GoProject\grpc-test-demo>grpcurl -plaintext  localhost:9999 list
grpc.reflection.v1alpha.ServerReflection
prod.ProductService

D:\GoProject\grpc-test-demo>grpcurl -plaintext   localhost:9999 describe prod.ProductService
Failed to resolve symbol "prod.ProductService": Symbol not found: prod.ProductService
caused by: File not found: status/status.proto

D:\GoProject\grpc-test-demo>grpcurl -plaintext -proto=go-grpc-proto/status/status.proto   localhost:9999 describe prod.ProductService
Failed to resolve symbol "prod.ProductService": Symbol not found: prod.ProductService

[jhump]

@tanjunchen, if you are using gogoproto (instead of the golang/protobuf library), then the issue you linked is most likely the underlying issue. Server reflection being coupled to the golang/protobuf library means that when compiled files using gogoproto instead, the descriptors aren't registered in a way that the golang/protobuf library can find them.

But if you are using normal protoc and --go_out to produce your Go code, then you are using already using golang/protobuf, so that wouldn't be the problem. In that case, it is most likely the same issue I described above -- a file has been compiled using a different relative path than by which it is imported. Above, the server cannot locate a descriptor for a file named status/status.proto. Most likely, that is how it appears in import statements. But it may have been compiled like protoc status.proto ... (e.g. the relative path given to protoc != the path used in import statements).

You would need to double-check how all files in the transitive closure of your service are compiled and make sure that the relative path given to protoc when compiling a given file exactly matches the relative file used for all import statements of that same file.

If you are using Go for your server, you might upgrade to v1.4 of the golang/protobuf runtime. IIUC, it will enforce that the relative paths are consistent and the program will fail to init if things are wrong.

[tanjunchen]

@jhump thanks . I guess that the problems are from the (e.g. the relative path given to protoc != the path used in import statements).

[ohenrik]

Could some one give a practical example of what @jhump explains? I can't get this to work irregardless of the folder structure or naming og the proto file, the import statements in the .proto file and the imports in the go files. Even if all of them are the same. Nothing is shown when I try to list packages or messages.

[jhump]

Nothing is shown when I try to list packages or messages.

@ohenrik, are you getting an error message? Or is it just showing nothing?

If the latter and just showing nothing, I think your server is not correctly setup with regards to the reflection service. I can't help too much on that unless you are using Go, since that is the main runtime whose service reflection feature is familiar to me. But in Go, there's usually not too much to actually configure -- it's supposed to be more or less automatic. One possible issue, if it's a Go service, could be if you are using gogo protobuf instead of the normal runtime, which has compatibility issues.

[ohenrik]

@jhump I'm using Go. Not sure If I'm using gogo proof, I installed the default following the gRPC basic tutorial website. My impression was also that this should work magically by just adding the reflection registration as per the documentation. But since there is no errors etc. I find it hard to debug. Is there a way to print anything inside the server it self to show what reflection has registered?

[jhuggart]

@jhump the info you've provided here is fantastic! Thanks for all of that. In my case, I think the relative path == the path given to protoc. But perhaps I'm wrong or missing something else.

I created a repo where I've reproduced the issue with the simplest version of the real problem I'm facing when trying to use reflection with a project that imports an external .proto. Would you mind taking a look when you get a chance?

Edit

Got it sorted out. I needed to change the file location and the go_package of the imported file. See this branch.

[jhump]

@jhuggart, looks like the problem is in your dependency.

You are importing the file as "github.com/danielvladco/go-proto-gql/pb/graphql.proto". But when that other repo generated code for their proto, they gave the path "pb/graphql.proto" to protoc (as seen here and here) to protoc. So, because of this mismatch, the service reflection code is unable to locate the descriptor for the import -- because the import in your file doesn't match the path used to register the dependency.

I would file an issue against their repo that they should generate the protos providing the same full path to protoc that others are expected to use in their import statements.

[mark4z]

everybody Tks.