Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate findCustom*Option functions in favor of extensions #669

Merged
merged 2 commits into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 51 additions & 136 deletions docs/writing_plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,7 @@ Code generator plugins can be created using the npm packages [@bufbuild/protoplu
- [Why use `f.import()`?](#why-use-fimport)
- [Exporting](#exporting)
- [Parsing plugin options](#parsing-plugin-options)
- [Reading custom options](#reading-custom-options)
- [Scalar options](#scalar-options)
- [Message options](#message-options)
- [Enum options](#enum-options)
- [Using custom Protobuf options](#using-custom-protobuf-options)
- [Testing](#testing)
- [Examples](#examples)

Expand Down Expand Up @@ -376,163 +373,81 @@ parseOption(key: string, value: string | undefined): void;

This function will be invoked by the framework, passing in any key/value pairs that it does not recognize from its pre-defined list.

### Reading custom options
### Using custom Protobuf options

Protobuf-ES does not yet provide support for extensions, neither in general as pertaining to proto2 nor with custom options in proto3. However, in the interim, there are convenience functions for retrieving any custom options specified in your .proto files. These are provided as a temporary utility until full extension support is implemented. There are three functions depending on the structure of the custom option desired (scalar, message, or enum):
Your plugin can support custom Protobuf options to modify the code it generates.

#### Scalar Options
As an example, let's use a custom service option to provide a default host.
Here is how this option would be used:

Custom options of a scalar type (`boolean`, `string`, `int32`, etc.) can be retrieved via the `findCustomScalarOption` function. It returns a type corresponding to the given `scalarType` parameter. For example, if `ScalarType.STRING` is passed, the return type will be a `string`. If the option is not found, it returns `undefined`.
```protobuf
syntax = "proto3";
import "customoptions/default_host.proto";
package connectrpc.eliza.v1;

```ts
function findCustomScalarOption<T extends ScalarType>(
desc: AnyDesc,
extensionNumber: number,
scalarType: T
): ScalarValue<T> | undefined;
```

`AnyDesc` represents any of the `DescXXX` objects such as `DescFile`, `DescEnum`, `DescMessage`, etc. The `extensionNumber` parameter represents the extension number of the custom options field definition.
service MyService {

The `scalarType` parameter is the type of the custom option you are searching for. `ScalarType` is an enum that represents all possible scalar types in the Protobuf grammar
// Set the default host for this service with our custom option.
option (customoptions.default_host) = "https://demo.connectrpc.com/";

For example, given the following:

```proto
extend google.protobuf.MessageOptions {
optional int32 foo_message_option = 50001;
}
extend google.protobuf.FieldOptions {
optional string foo_field_option = 50002;
// ...
}

message FooMessage {
option (foo_message_option) = 1234;

int32 foo = 1 [(foo_field_option) = "test"];
}
```

The values of these options can be retrieved as follows:

```ts
const msgVal = findCustomScalarOption(descMessage, 50001, ScalarType.INT32); // 1234

const fieldVal = findCustomScalarOption(descField, 50002, ScalarType.STRING); // "test"
```

#### Message Options

Custom options of a more complex message type can be retrieved via the `findCustomMessageOption` function. It returns a concrete type with fields populated corresponding to the values set in the proto file.

```ts
export function findCustomMessageOption<T extends Message<T>>(
desc: AnyDesc,
extensionNumber: number,
msgType: MessageType<T>
): T | undefined {
```
Custom options are extensions to one of the options messages defined in
`google/protobuf/descriptor.proto`. Here is how we can define the option we are
using above:

The `msgType` parameter represents the type of the message you are searching for.
```protobuf
// customoptions/default_host.proto
syntax = "proto3";
import "google/protobuf/descriptor.proto";
package customoptions;

For example, given the following proto files:

```proto
// custom_options.proto

extend google.protobuf.MethodOptions {
optional ServiceOptions service_method_option = 50007;
}

message ServiceOptions {
int32 foo = 1;
string bar = 2;
oneof qux {
string quux = 3;
}
repeated string many = 4;
map<string, string> mapping = 5;
extend google.protobuf.ServiceOptions {
// We extend the ServiceOptions message, so that other proto files can import
// this file, and set the option on a service declaration.
optional string default_host = 1001;
}
```

```proto
// service.proto
You can learn more about custom options in the [language guide](https://protobuf.dev/programming-guides/proto3/#customoptions).

import "custom_options.proto";

service FooService {
rpc Get(GetRequest) returns (GetResponse) {
option (service_method_option) = {
foo: 567,
bar: "Some string",
quux: "Oneof string",
many: ["a", "b", "c"],
mapping: [{key: "testKey", value: "testVal"}]
};
}
}
```

You can retrieve the options using a generated type by first generating the file which defines the custom option type. Then, import and pass this type to the `findCustomMessageOption` function.
First, we need to generate code for our custom option. This will generate a file
`customoptions/default_host_pb.ts` with a new export - our extension:

```ts
import { ServiceOptions } from "./gen/proto/custom_options_pb.js";

const option = findCustomMessageOption(method, 50007, ServiceOptions)

console.log(option);
/*
* {
* foo: 567,
* bar: "Some string",
* quux: "Oneof string",
* many: ["a", "b", "c"],
* mapping: [{key: "testKey", value: "testVal"}]
* }
*/
```
import { ServiceOptions, Extension } from "@bufbuild/protobuf";

Note that `repeated` and `map` values are only supported within a custom message option. They are not supported as option types independently. If you have need to use a custom option that is `repeated` or is of type `map`, it is recommended to use a message option as a wrapper.


#### Enum Options

Custom options of an enum type can be retrieved via the `findCustomEnumOption` function. It returns a `number` corresponding to the `enum` value set in the option.

```ts
export function findCustomEnumOption(
desc: AnyDesc,
extensionNumber: number
): number | undefined {
export const default_host: Extension<ServiceOptions, string> = ...
```

The returned number can then be coerced into the concrete `enum` type. The `enum` type just needs to be generated ahead of time much like the example in `findCustomMessageOption`.
Now we can utilize this extension to read custom options in our plugin:

For example, given the following:

```proto
extend google.protobuf.MessageOptions {
optional FooEnum foo_enum_option = 50001;
}

enum FooEnum {
UNDEFINED = 0;
ACTIVE = 1;
INACTIVE = 2;
}

message FooMessage {
option (foo_enum_option) = ACTIVE;

string name = 1;
```ts
import type { GeneratedFile } from "@bufbuild/protoplugin/ecmascript";
import { ServiceOptions, ServiceDesc, hasExtension, getExtension } from "@bufbuild/protobuf";
import { default_host } from "./customoptions/default_host_pb.js";

function generateService(desc: ServiceDesc, f: GeneratedFile) {
// The protobuf message google.protobuf.ServiceOptions contains our custom
// option.
const serviceOptions: ServiceOptions | undefined = service.proto.options;

// Let's see if our option was set:
if (serviceOptions && hasExtension(serviceOption, default_host)) {
const value = getExtension(serviceOption, default_host); // "https://demo.connectrpc.com/"
// Our option was set, we can use it here.
}
}
```

The value of this option can be retrieved as follows:
Custom options can be set on any Protobuf element. They can be simple singular
string fields as the one above, but also repeated fields, message fields, etc.

Take a look at our [plugin example](https://github.com/bufbuild/protobuf-es/tree/main/packages/protoplugin-example)
to see the custom option above in action, and run the code yourself.

```ts
const enumVal: FooEnum | undefined = findCustomEnumOption(descMessage, 50001); // FooEnum.ACTIVE
```

## Testing

Expand Down
3 changes: 1 addition & 2 deletions packages/protoplugin-example/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@
"build": "../../node_modules/typescript/bin/tsc --noEmit",
"start": "npx esbuild src/index.ts --serve=localhost:3000 --servedir=www --outdir=www --bundle --global-name=eliza",
"test": "tsx --test test/*.ts",
"pregenerate": "rm -rf src/gen",
"generate": "buf generate buf.build/connectrpc/eliza"
"generate": "buf generate proto"
},
"license": "Apache-2.0",
"dependencies": {
Expand Down
41 changes: 41 additions & 0 deletions packages/protoplugin-example/proto/connectrpc/eliza.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// Copyright 2021-2024 Buf Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

import "customoptions/default_host.proto";

package connectrpc.eliza.v1;

// This is a modified copy of ElizaService from https://buf.build/connectrpc/eliza
service ElizaService {

// Set the default host for this service with a custom option.
// Comment this line out and run `npm run generate` to see how the generated
// code changes.
option (customoptions.default_host) = "https://demo.connectrpc.com/";

// Say is a unary RPC. Eliza responds to the prompt with a single sentence.
rpc Say(SayRequest) returns (SayResponse) {
option idempotency_level = NO_SIDE_EFFECTS;
}
}
// SayRequest is a single-sentence request.
message SayRequest {
string sentence = 1;
}
// SayResponse is a single-sentence response.
message SayResponse {
string sentence = 1;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
// Copyright 2021-2024 Buf Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

import "google/protobuf/descriptor.proto";

package customoptions;

extend google.protobuf.ServiceOptions {

// An example for a custom option. Custom options are extensions to one of
// the options messages defined in descriptor.proto.
//
// We extend the ServiceOptions message, so that other proto files can import
// this file, and set the option on a service declaration.
optional string default_host = 1001;

}
Loading