When using quarkus.generate-code.grpc.scan-for-proto-include to build classes for subset of proto files from dependency, proto imports fail

[basselworkforce]

Describe the bug

When scanning a dependency for proto files to generate code for e.g.: quarkus.generate-code.grpc.scan-for-proto=com.acme:proto-sources-library and picking a subset of proto files to generate code for e.g. quarkus.generate-code.grpc.scan-for-proto-include."com.acme:proto-sources-library"=**/orders/**, code gen fails when the proto files have import statements relative to the root path.

For example:

com/acme/grpc/orders/v1/order.proto file

syntax = "proto3";

package com.acme.grpc.orders.v1;

import "com/acme/grpc/orders/v1/order_status.proto";

message Order {
  string id = 1;
  OrderStatus status = 2;
}

com/acme/grpc/orders/v1/order_status.proto file

syntax = "proto3";

package com.acme.grpc.orders.v1;

message OrderStatus {
  string id = 1;
}

Run
./gradlew quarkusGenerateCodeDev

Observe error

com/acme/grpc/orders/v1/order_status.proto: File not found.
order.proto:5:1: Import "com/acme/grpc/orders/v1/order_status.proto" was not found or had errors.
order.proto:9:3: "OrderStatus" is not defined.
> Task :quarkusGenerateCodeDev FAILED

Stack trace

Caused by: io.quarkus.bootstrap.prebuild.CodeGenException: Failed to generate Java classes from proto files: [/home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f/com/acme/grpc/orders/v1/order_status.proto, /home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f/com/acme/grpc/orders/v1/order.proto.proto] to /home/brachid/orders/build/classes/java/quarkus-generated-sources/grpc with command /home/brachid/orders/build/com.google.protobuf-protoc-linux-x86_64-exe -I=/home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f/com/acme/grpc/orders/v1 -I=/home/brachid/orders/build/protoc-dependencies/887869d68b3d4324376eb9ee6db4956483e8eaf8 --plugin=protoc-gen-grpc=/home/brachid/orders/build/io.grpc-protoc-gen-grpc-java-linux-x86_64-exe --plugin=protoc-gen-q-grpc=/home/brachid/orders/build/quarkus-grpc10919188769645557693.sh --q-grpc_out=/home/brachid/orders/build/classes/java/quarkus-generated-sources/grpc --grpc_out=/home/brachid/orders/build/classes/java/quarkus-generated-sources/grpc --java_out=/home/brachid/orders/build/classes/java/quarkus-generated-sources/grpc /home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f/com/acme/grpc/orders/v1/order_status.proto /home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f/com/acme/grpc/orders/v1/order.proto

It seems the problem is that the correct protoc import directory (-I or --proto_path) argument is not one of the two passed in:
com.google.protobuf-protoc-linux-x86_64-exe -I=/home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f/com/acme/grpc/orders/v1 -I=/home/brachid/orders/build/protoc-dependencies/887869d68b3d4324376eb9ee6db4956483e8eaf8 ...

The correct import directory argument would be: -I=/home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f which is the root of the proto files. The import statements use a path relative to that root as per the recommendation in the guidelines:

In general you should set the --proto_path flag to the root of your project and use fully qualified names for all imports.
https://protobuf.dev/programming-guides/proto3/#importing

If I re-run the protoc command with the correct -I argument added on then it succeeds:
com.google.protobuf-protoc-linux-x86_64-exe -I=/home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f -I=/home/brachid/orders/build/protoc-protos-from-dependencies/58d495f113cf26357f150a443d3b533cca56d58f/com/acme/grpc/orders/v1 -I=/home/brachid/orders/build/protoc-dependencies/887869d68b3d4324376eb9ee6db4956483e8eaf8 ...

Expected behavior

Code gen should succeed

Actual behavior

Code gen fails on the protoc command

How to Reproduce?

No response

Output of uname -a or ver

Linux 5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

openjdk version "21.0.4" 2024-07-16 LTS

Quarkus version or git rev

3.15.1

Build tool (ie. output of mvnw --version or gradlew --version)

Gradle 8.9

Additional information

No response

[quarkus-bot]

/cc @alesj (grpc), @cescoffier (grpc)

[cescoffier]

Thanks!

Can you provide a simple reproducer? You already gave the protoc command and that's great!

We would like to have a larger tests regarding the gRPC import.

[basselworkforce]

done: https://github.com/basselworkforce/proto-consumer

also this is the proto source code for reference: https://github.com/basselworkforce/proto-sources-library
and the published dependency is here: https://github.com/basselworkforce/mvn-repo/packages/2265803

[cescoffier]

Thanks @basselworkforce.

@alesj can you have a look?

[basselworkforce]

Took a quick stab at this here: #43706, seems to fix the issue when I tested locally

[cescoffier]

Thanks for the feedback! Much appreciated.

[basselworkforce]

No problem!