Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

L70: @grpc/proto-loader TypeScript Type Generator CLI Tool #183

Merged
merged 11 commits into from
Jul 21, 2020
65 changes: 65 additions & 0 deletions L70-node-proto-loader-type-generator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
@grpc/proto-loader TypeScript Type Generator CLI Tool
----
* Author(s): murgatroid99
* Approver: wenbozhu
* Status: Draft
* Implemented in: Node.js
* Last updated: 2020-06-01
* Discussion at: https://groups.google.com/g/grpc-io/c/FwLprMC-5Qc

## Abstract

Add a tool to the `@grpc/proto-loader` to generate types that describe the objects that will be generated by loading the output of `@grpc/proto-loader` into `grpc` or `@grpc/grpc-js`.

## Background

`@grpc/proto-loader` outputs objects with types determined by the `.proto` files loaded at runtime, so it is currently very difficult to get compile-time type information about those objects. As TypeScript becomes increasingly popular, that compile-time type information becomes increasingly desirable.


### Related Proposals:
* L23 Standalone Node.js gRPC+Protobuf.js Library API

## Proposal

In the `@grpc/proto-loader` library, provide a command line tool that will generate the types of the objects that will be created by passing the output of `load` or `loadSync` to `grpc.loadPackageDefinition` when loading a specific set of `.proto` files with a specific set of options. This will allow users to write code like this to use the type information in the rest of the code:

```ts
import * as grpc from 'grpc';
// OR
import * as grpc from '@grpc/grpc-js';

import ProtoGrpcType from 'proto-file-name.d.ts'; // File generated by the tool
murgatroid99 marked this conversation as resolved.
Show resolved Hide resolved
import * as protoLoader from '@grpc/proto-loader';

const packageDefinition = protoLoader.loadSync('proto-file-name.proto', options);
const loadedPackageDefinition = grpc.loadPackageDefinition(packageDefinition) as ProtoGrpcType<grpc.Client, grpc.CallOptions>; // How the types will be used
```

The tool will be named `proto-loader-gen-types` and will have this usage:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this work as a protoc plugin? Or will it stand on it's own.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a standalone tool, distributed in the @grpc/proto-loader package. It uses Protobuf.js rather than protoc for parsing.


```
proto-loader-gen-types [OPTIONS] <proto file name>
```

The options will correspond directly to the `protoLoader.load` options as follows:

- `--keep-case`: Preserve the case of field names
- `--longs=String|Number`: Specify the type that should be used to output 64 bit integer values
- `--enums=String`: Specify that enum values should be output as strings
- `--bytes=Array|String`: Specify the type that should be used to output `bytes` fields
- `--defaults`: Indicates that default values should be output for missing fields
- `--arrays`: Indicates that empty arrays should be output for missing repeated fields even if `--defaults` is unset
- `--objects`: Indicates that empty objects should be output for missing message fields even if `--defaults` is unset
- `--oneofs`: Indicates that virtual "oneof" fields will be set to the present field's name in the output
- `--includeDirs=<directories...>`, `-I`: A list of directories to search for included `.proto` files

## Rationale

The goal is to have type information available when editing and building code that uses `@grpc/proto-loader` to load `.proto` files at runtime. It is not possible to infer this type information from the interfaces currently provided by this library because TypeScript cannot infer type information from `.proto` files. The only other option is to generate this type information separately. The recommended usage of the `@grpc/proto-loader` library is to call `load` or `loadSync` and then pass the result of that to `grpc.loadPackageDefinition`, so the type of the final result of that will be useful to most users. The output of `load` and `loadSync` are objects with runtime data that describes the resulting types, so the types of those objects are significantly less useful, and the final type that we will generate here cannot be effectively inferred from those types.

The type parameters `grpc.Client`, `grpc.Metadata`, and `grpc.CallOptions` are all needed because each of those types is used in the resulting type, and none of them can be inferred from the others. The type `grpc.Metadata` also impacts the resulting type, but its type is the same across the two implementations so it can be defined independently of them.


## Implementation

I (murgatroid99) will implement this in the `@grpc/proto-loader` library.