Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update guidance on how to write a plugin #1218

Merged
merged 2 commits into from
Nov 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions docs/plugin-authoring-evaluator.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,13 +262,14 @@ In this case we have two evaluators `DELICIOUSNESS` and `US_PHONE_REGEX_MATCH`.

```ts
export function myAwesomeEval<ModelCustomOptions extends z.ZodTypeAny>(
params: PluginOptions<ModelCustomOptions>
options: PluginOptions<ModelCustomOptions>
): PluginProvider {
// Define the new plugin
const plugin = genkitPlugin(
const plugin = (options?: MyPluginOptions<ModelCustomOptions>) => {
return genkitPlugin(
'myAwesomeEval',
async (params: PluginOptions<ModelCustomOptions>) => {
const { judge, judgeConfig, metrics } = params;
async (ai: Genkit) => {
const { judge, judgeConfig, metrics } = options;
const evaluators: EvaluatorAction[] = metrics.map((metric) => {
// We'll create these functions in the next step
switch (metric) {
Expand All @@ -281,11 +282,10 @@ export function myAwesomeEval<ModelCustomOptions extends z.ZodTypeAny>(
}
});
return { evaluators };
}
);

// Create the plugin with the passed params
return plugin(params);
})
}
// Create the plugin with the passed options
return plugin(options);
}
export default myAwesomeEval;
```
Expand All @@ -299,7 +299,7 @@ For evaluation with Gemini, disable safety settings so that the evaluator can ac
```ts
import { gemini15Flash } from '@genkit-ai/googleai';

export default configureGenkit({
const ai = genkit({
plugins: [
...
myAwesomeEval({
Expand Down
21 changes: 13 additions & 8 deletions docs/plugin-authoring-telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,19 @@ Logging is provided separately so that a plugin can control where the data is
written explicitly.

```ts
import { genkitPlugin, Plugin } from '@genkit-ai/core';
import { Genkit } from 'genkit';
import { genkitPlugin, GenkitPlugin } from'genkit/plugin';


...

export interface MyPluginOptions {
// [Optional] Your plugin options
}

export const myPlugin: Plugin<[MyPluginOptions] | []> = genkitPlugin(
export const myPlugin: GenkitPlugin<[MyPluginOptions] | []> = genkitPlugin(
'myPlugin',
async (options?: MyPluginOptions) => {
async (ai: Genkit) => {
return {
telemetry: {
instrumentation: {
Expand Down Expand Up @@ -226,6 +228,7 @@ The following is a full example of the telemetry plugin created above. For
a real world example, take a look at the `@genkit-ai/google-cloud` plugin.

```ts
import { genkitPlugin, GenkitPlugin } from 'genkit/plugin';
import {
genkitPlugin,
LoggerConfig,
Expand Down Expand Up @@ -293,9 +296,11 @@ const myLogger: LoggerConfig = {
},
};

export const myPlugin: Plugin<[MyPluginOptions] | []> = genkitPlugin(
'myPlugin',
async (options?: MyPluginOptions) => {
export function myPlugin(options?: MyPluginOptions) {
return genkitPlugin('myPlugin', async (ai: Genkit) => {
ai.defineModel(...);
ai.defineEmbedder(...)
// ....
return {
telemetry: {
instrumentation: {
Expand All @@ -308,8 +313,8 @@ export const myPlugin: Plugin<[MyPluginOptions] | []> = genkitPlugin(
},
},
};
}
);
});
};

export default myPlugin;
```
Expand Down
102 changes: 58 additions & 44 deletions docs/plugin-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ cd genkitx-my-plugin

npm init -y

npm i --save @genkit-ai/core
npm i --save genkit

npm i --save-dev typescript

Expand All @@ -46,12 +46,17 @@ interface MyPluginOptions {
// add any plugin configuration here
}

export const myPlugin = genkitPlugin(
'my-plugin',
async (options: MyPluginOptions) => {
// initialize your plugin here...
}
);
import { Genkit, z } from 'genkit';
import { GenkitPlugin, genkitPlugin } from 'genkit/plugin';

export function myPlugin(options?: MyPluginOptions) {
return genkitPlugin('myPlugin', async (ai: Genkit) => {
ai.defineModel(...);
ai.defineEmbedder(...)
// ....
});
};

```

### Plugin options guidance
Expand All @@ -62,26 +67,30 @@ requires a secret value, such as API keys, you should offer both an option and a
default environment variable to configure it:

```ts
import { genkitPlugin, GenkitError } from '@genkit-ai/core';
import { Genkit, z } from 'genkit';
import { GenkitPlugin, genkitPlugin } from 'genkit/plugin';
import { GenkitError } from '@genkit-ai/core';

interface MyPluginOptions {
apiKey?: string;
}

export const myPlugin = genkitPlugin(
'my-plugin',
async (options: MyPluginOptions) => {
const apiKey = options.apiKey || process.env.MY_PLUGIN_API_KEY;
export function myPlugin(options?: MyPluginOptions) {
return genkitPlugin('myPlugin', async (ai: Genkit) => {
if (!apiKey)
throw new GenkitError({
source: 'my-plugin',
status: 'INVALID_ARGUMENT',
message:
'Must supply either `options.apiKey` or set `MY_PLUGIN_API_KEY` environment variable.',
});
// ... continue initialization
}
);

ai.defineModel(...);
ai.defineEmbedder(...)

// ....
});
};
```

## Building your plugin
Expand Down Expand Up @@ -110,39 +119,44 @@ npm i --save @genkit-ai/ai
At a high level, a model plugin might look something like this:

```ts
import { genkitPlugin, GenkitError } from '@genkit-ai/core';
import { defineModel, GenerationCommonConfigSchema } from '@genkit-ai/ai/model';
import { genkitPlugin, GenkitPlugin } from 'genkit/plugin';
import { GenkitError } from '@genkit-ai/core';
import { GenerationCommonConfigSchema } from '@genkit-ai/ai/model';
import { simulateSystemPrompt } from '@genkit-ai/ai/model/middleware';
import { z } from 'zod';

export const myPlugin = genkitPlugin('my-plugin', async (options: {apiKey?: string}) => {
defineModel({
// be sure to include your plugin as a provider prefix
name: 'my-plugin/my-model',
// label for your model as shown in Genkit Developer UI
label: 'My Awesome Model',
// optional list of supported versions of your model
versions: ['my-model-001', 'my-model-001'],
// model support attributes
supports: {
multiturn: true, // true if your model supports conversations
media: true, // true if your model supports multimodal input
tools: true, // true if your model supports tool/function calling
systemRole: true, // true if your model supports the system role
output: ['text', 'media', 'json'], // types of output your model supports
},
// Zod schema for your model's custom configuration
configSchema: GenerationCommonConfigSchema.extend({
safetySettings: z.object({...}),
}),
// list of middleware for your model to use
use: [simulateSystemPrompt()]
}, async request => {
const myModelRequest = toMyModelRequest(request);
const myModelResponse = await myModelApi(myModelRequest);
return toGenerateResponse(myModelResponse);
export function myPlugin(options?: MyPluginOptions) {
return genkitPlugin('my-plugin', async (ai: Genkit) => {
ai.defineModel({
// be sure to include your plugin as a provider prefix
name: 'my-plugin/my-model',
// label for your model as shown in Genkit Developer UI
label: 'My Awesome Model',
// optional list of supported versions of your model
versions: ['my-model-001', 'my-model-001'],
// model support attributes
supports: {
multiturn: true, // true if your model supports conversations
media: true, // true if your model supports multimodal input
tools: true, // true if your model supports tool/function calling
systemRole: true, // true if your model supports the system role
output: ['text', 'media', 'json'], // types of output your model supports
},
// Zod schema for your model's custom configuration
configSchema: GenerationCommonConfigSchema.extend({
safetySettings: z.object({...}),
}),
// list of middleware for your model to use
use: [simulateSystemPrompt()]
}, async request => {
const myModelRequest = toMyModelRequest(request);
const myModelResponse = await myModelApi(myModelRequest);
return toGenerateResponse(myModelResponse);
});
});
});
};


```

#### Transforming Requests and Responses
Expand Down
Loading