Extensibility / Plugin Support

[mightyguava]

Add support for calling into plugins during code generation. One specific use case I have is to support automatic encryption/decryption of columns. To illustrate, here's very naive proposal:

CREATE TABLE authors (
          id   BIGSERIAL PRIMARY KEY,
          name text      NOT NULL, -- :encrypted
          bio  text -- :encrypted
);

Adding the :encrypted annotation and having the encryption plugin enabled would cause sqlc to augment the generated code to look like the below:

// db.go
interface Cipher {
  Encrypt(s string) (string, error)
  Decrypt(s string) (string, error)
}

func New(db DBTX) *Queries {
	return &Queries{db: db, enc: &NoopCipher{}}
}

func NewWithEncryptor(db DBTX, ciph Cipher) *Queries {
	return &Queries{db: db, ciph: ciph}
}

type Queries struct {
	db DBTX
	ciph Cipher
}

// query.sql.go
const createAuthor = `-- name: CreateAuthor :one
INSERT INTO authors (
          name, bio
) VALUES (
  $1, $2
)
RETURNING id, name, bio
`

type CreateAuthorParams struct {
	Name string
	Bio  sql.NullString
}

func (q *Queries) CreateAuthor(ctx context.Context, arg CreateAuthorParams) (Author, error) {
	var err error
	var i Author

	arg.Name, err = q.ciph.Encrypt(arg.Bio)
	if err != nil {
		return i, err
	}
	arg.Bio, err = q.ciph.Encrypt(arg.Bio)
	if err != nil {
		return i, err
	}
	row := q.db.QueryRowContext(ctx, createAuthor, arg.Name, arg.Bio)
	err := row.Scan(&i.ID, &i.Name, &i.Bio)
	if err != nil {
		return i, err
	}
	i.Name, err = q.ciph.Decrypt(i.Name)
	if err != nil {
		return i, err
	}
	i.Bio, err = q.ciph.Decrypt(i.Bio)
	if err != nil {
		return i, err
	}

	return i, err
}

The Cipher is be an interface dictated by the plugin that the user would have to provide.

This proposal leaves a lot to answer and is just an example of what we can do with plugins. sqlc having an understanding of the schema and being able to parse annotations (comments) makes it a great place to hook into this automated code for translating. I don't think this is achievable simply with a custom type, since the Cipher needs to be initialized with secret key material to perform the encryption/decryption.

I don't know how exactly you would do a plugin with sqlc. Maybe make it easy to create a custom build of the sqlc CLI, and adding support to the library for code generation hooks. Or, maybe invoking external binaries that communicate over grpc (a pattern in the Go community nowadays), but that could result in a complex protocol and you'd have to describe the AST and in-memory models in protobuf.

I think this could be a really powerful feature though. Curious what your thoughts are.

[kyleconroy]

I think this could be a really powerful feature though. Curious what your thoughts are.

sqlc isn't at a place where plugins or extensions make sense. Long term, I'd like to have first-class support for extending sqlc to do cool things like encryption. Right now the internals are under heavy development and we haven't even decided on the Go interface between code generation, query parsing, and type checking.

[kyleconroy]

👀 #1414 👀

[kyleconroy]

#1684