(feature): Add Protobuf mapper types

[amckinney]

Overview

This adds support for gRPC/Protobuf types in IRv53. With this, generators can support mapping Fern-generated types to/from their Protobuf equivalent representation defined in the user's .proto file. This includes updating the IR, YAML definition schema, generator configuration, and OpenAPI extensions.

Example

Consider the following Protobuf service definition:

// proto/user/v1/user.proto

syntax = "proto3";

package user.v1;

import "google/api/annotations.proto";
import "google/api/field_behavior.proto";
import "google/protobuf/struct.proto";

option csharp_namespace = "User.V1";

message User {
    string username = 1; 
    string email = 2;
    uint age = 3;
    float weight = 4;
    google.protobuf.Struct metadata = 5;
}

message CreateUserRequest {
    string username = 1;
    string email = 2;
    uint age = 3;
    float weight = 4;
    google.protobuf.Struct metadata = 5;
}

message CreateUserResponse {
    User user = 1;
}

service UserService {
  rpc CreateUser(CreateUserRequest) returns (CreateResponse) {
    option (google.api.http) = {
      post: "/users"
      body: "*"
    };
  }
}

We now expose a few new configuration options to annotate the Fern definition with the context necessary to map to and from the message types defined in proto/user/v1/user.proto. In combination, we have a Fern definition that looks like the following:

# generators.yml
api:
  - proto:
       root: proto
       target: proto/user/v1/user.proto

Behind the scenes, this is equivalent to a Fern definition that looks like the following:

types:
  Metadata:
    type: map<string, optional<MetadataValue>>
    encoding:
      proto:
        type: google.protobuf.Struct

  MetadataValue:
    discriminated: false
    union:
      - double
      - string
      - boolean
      - list<MetadataValue>
    encoding:
      proto:
        type: google.protobuf.Value

  User:
    properties:
      id: string
      username: string
      email: optional<string>
      age: optional<uint>
      weight: optional<float>
      metadata: optional<Metadata>

  CreateUserResponse:
    properties:
      user: User

service:
  auth: false
  base-path: /
  transport:
    grpc:
      service-name: UserService
  endpoints:
    createUser:
      method: POST
      path: /users
      request:
        name: CreateUserRequest
        body:
          properties:
            username: string
            email: optional<string>
            age: optional<uint>
            weight: optional<float>
      response: CreateUserResponse

[dsinghvi]

I think the reality is that the source can be multiple OpenAPI specs, FernDefinitions or Proto files. Thoughts on making this:

sources: 
  type: list<ApiDefinitionSource>
  docs: The raw API definitions that produced the IR 
  
ApiDefinitionSource: 
  union: 
    proto: ProtoSource
    openapi: OpenAPISource # can skip for now

[dsinghvi]

The C# generator can just handle processing a single (the first) ProtoSource for now

[dsinghvi]

Also would be great if each source had a unique ApiDefinitionSourceId (so that in the future we can mark each type / endpoint with a particular source)

[amckinney]

Nice call - the ProtoSourceConfig as-is handles the multiple .proto files (via the root directory URL reference), but it doesn't include mixing and matching between Fern / OpenAPI / Proto.

I wonder if it's worth still wrapping sources in an object so it's easy to extend later. Thoughts on this mild adjustment?

SourceConfig:
  properties: 
    sources:
      docs: The raw API definitions that produced the IR.
      type: list<ApiDefinitionSource>
  
ApiDefinitionSource:
  union:
    proto: ProtoSource
    openapi: {}

[dsinghvi]

Don't see any logic that actually pushes thes extensions into the fern definition. is that intentional?

[amckinney]

Yes, nothing implemented just yet - just spec'ing out the interface for now. I'll incorporate this when they're used in a follow-up.