Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(assistants): add support for streaming #714

Merged
merged 1 commit into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 49 additions & 9 deletions api.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

Types:

- <code><a href="./src/resources/shared.ts">ErrorObject</a></code>
- <code><a href="./src/resources/shared.ts">FunctionDefinition</a></code>
- <code><a href="./src/resources/shared.ts">FunctionParameters</a></code>

Expand Down Expand Up @@ -177,6 +178,15 @@ Types:

- <code><a href="./src/resources/beta/assistants/assistants.ts">Assistant</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">AssistantDeleted</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">AssistantStreamEvent</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">AssistantTool</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">CodeInterpreterTool</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">FunctionTool</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">MessageStreamEvent</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">RetrievalTool</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">RunStepStreamEvent</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">RunStreamEvent</a></code>
- <code><a href="./src/resources/beta/assistants/assistants.ts">ThreadStreamEvent</a></code>

Methods:

Expand Down Expand Up @@ -214,6 +224,7 @@ Methods:
- <code title="post /threads/{thread_id}">client.beta.threads.<a href="./src/resources/beta/threads/threads.ts">update</a>(threadId, { ...params }) -> Thread</code>
- <code title="delete /threads/{thread_id}">client.beta.threads.<a href="./src/resources/beta/threads/threads.ts">del</a>(threadId) -> ThreadDeleted</code>
- <code title="post /threads/runs">client.beta.threads.<a href="./src/resources/beta/threads/threads.ts">createAndRun</a>({ ...params }) -> Run</code>
- <code>client.beta.threads.<a href="./src/resources/beta/threads/threads.ts">createAndRunStream</a>(body, options?) -> AssistantStream</code>

### Runs

Expand All @@ -231,16 +242,29 @@ Methods:
- <code title="get /threads/{thread_id}/runs">client.beta.threads.runs.<a href="./src/resources/beta/threads/runs/runs.ts">list</a>(threadId, { ...params }) -> RunsPage</code>
- <code title="post /threads/{thread_id}/runs/{run_id}/cancel">client.beta.threads.runs.<a href="./src/resources/beta/threads/runs/runs.ts">cancel</a>(threadId, runId) -> Run</code>
- <code title="post /threads/{thread_id}/runs/{run_id}/submit_tool_outputs">client.beta.threads.runs.<a href="./src/resources/beta/threads/runs/runs.ts">submitToolOutputs</a>(threadId, runId, { ...params }) -> Run</code>
- <code>client.beta.threads.runs.<a href="./src/resources/beta/threads/runs/runs.ts">createAndStream</a>(threadId, body, options?) -> AssistantStream</code>
- <code>client.beta.threads.runs.<a href="./src/resources/beta/threads/runs/runs.ts">submitToolOutputsStream</a>(threadId, runId, body, options?) -> AssistantStream</code>

#### Steps

Types:

- <code><a href="./src/resources/beta/threads/runs/steps.ts">CodeToolCall</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">CodeInterpreterLogs</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">CodeInterpreterOutputImage</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">CodeInterpreterToolCall</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">CodeInterpreterToolCallDelta</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">FunctionToolCall</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">FunctionToolCallDelta</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">MessageCreationStepDetails</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">RetrievalToolCall</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">RetrievalToolCallDelta</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">RunStep</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">RunStepDelta</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">RunStepDeltaEvent</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">RunStepDeltaMessageDelta</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">ToolCall</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">ToolCallDelta</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">ToolCallDeltaObject</a></code>
- <code><a href="./src/resources/beta/threads/runs/steps.ts">ToolCallsStepDetails</a></code>

Methods:
Expand All @@ -252,17 +276,33 @@ Methods:

Types:

- <code><a href="./src/resources/beta/threads/messages/messages.ts">MessageContentImageFile</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">MessageContentText</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">ThreadMessage</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">ThreadMessageDeleted</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">Annotation</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">AnnotationDelta</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">FileCitationAnnotation</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">FileCitationDeltaAnnotation</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">FilePathAnnotation</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">FilePathDeltaAnnotation</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">ImageFile</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">ImageFileContentBlock</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">ImageFileDelta</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">ImageFileDeltaBlock</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">Message</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">MessageContent</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">MessageContentDelta</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">MessageDeleted</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">MessageDelta</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">MessageDeltaEvent</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">Text</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">TextContentBlock</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">TextDelta</a></code>
- <code><a href="./src/resources/beta/threads/messages/messages.ts">TextDeltaBlock</a></code>

Methods:

- <code title="post /threads/{thread_id}/messages">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">create</a>(threadId, { ...params }) -> ThreadMessage</code>
- <code title="get /threads/{thread_id}/messages/{message_id}">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">retrieve</a>(threadId, messageId) -> ThreadMessage</code>
- <code title="post /threads/{thread_id}/messages/{message_id}">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">update</a>(threadId, messageId, { ...params }) -> ThreadMessage</code>
- <code title="get /threads/{thread_id}/messages">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">list</a>(threadId, { ...params }) -> ThreadMessagesPage</code>
- <code title="post /threads/{thread_id}/messages">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">create</a>(threadId, { ...params }) -> Message</code>
- <code title="get /threads/{thread_id}/messages/{message_id}">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">retrieve</a>(threadId, messageId) -> Message</code>
- <code title="post /threads/{thread_id}/messages/{message_id}">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">update</a>(threadId, messageId, { ...params }) -> Message</code>
- <code title="get /threads/{thread_id}/messages">client.beta.threads.messages.<a href="./src/resources/beta/threads/messages/messages.ts">list</a>(threadId, { ...params }) -> MessagesPage</code>

#### Files

Expand Down
39 changes: 39 additions & 0 deletions examples/assistant-stream-raw.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import OpenAI from 'openai';

const openai = new OpenAI();

async function main() {
const assistant = await openai.beta.assistants.create({
model: 'gpt-4-1106-preview',
name: 'Math Tutor',
instructions: 'You are a personal math tutor. Write and run code to answer math questions.',
});

const thread = await openai.beta.threads.create({
messages: [
{
role: 'user',
content: '"I need to solve the equation `3x + 11 = 14`. Can you help me?"',
},
],
});

const stream = await openai.beta.threads.runs.create(thread.id, {
assistant_id: assistant.id,
additional_instructions: 'Please address the user as Jane Doe. The user has a premium account.',
stream: true,
});

for await (const event of stream) {
if (event.event === 'thread.message.delta') {
const chunk = event.data.delta.content?.[0];
if (chunk && 'text' in chunk) {
process.stdout.write(chunk.text.value);
}
}
}

console.log();
}

main();
48 changes: 48 additions & 0 deletions examples/assistant-stream.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/usr/bin/env -S npm run tsn -T

import OpenAI from 'openai';

/**
* Example of streaming a response from an assistant
*/

const openai = new OpenAI();

async function main() {
const assistant = await openai.beta.assistants.create({
model: 'gpt-4-1106-preview',
name: 'Math Tutor',
instructions: 'You are a personal math tutor. Write and run code to answer math questions.',
});

let assistantId = assistant.id;
console.log('Created Assistant with Id: ' + assistantId);

const thread = await openai.beta.threads.create({
messages: [
{
role: 'user',
content: '"I need to solve the equation `3x + 11 = 14`. Can you help me?"',
},
],
});

let threadId = thread.id;
console.log('Created thread with Id: ' + threadId);

const run = openai.beta.threads.runs
.createAndStream(threadId, {
assistant_id: assistantId,
})
//Subscribe to streaming events and log them
.on('event', (event) => console.log(event))
.on('textDelta', (delta, snapshot) => console.log(snapshot))
.on('messageDelta', (delta, snapshot) => console.log(snapshot))
.on('run', (run) => console.log(run))
.on('messageDelta', (delta, snapshot) => console.log(snapshot))
.on('connect', () => console.log());
const result = await run.finalRun();
console.log('Run Result' + result);
}

main();
57 changes: 57 additions & 0 deletions examples/assistants.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env -S npm run tsn -T

import OpenAI from 'openai';
import { sleep } from 'openai/core';

/**
* Example of polling for a complete response from an assistant
*/

const openai = new OpenAI();

async function main() {
const assistant = await openai.beta.assistants.create({
model: 'gpt-4-1106-preview',
name: 'Math Tutor',
instructions: 'You are a personal math tutor. Write and run code to answer math questions.',
// tools = [],
});

let assistantId = assistant.id;
console.log('Created Assistant with Id: ' + assistantId);

const thread = await openai.beta.threads.create({
messages: [
{
role: 'user',
content: '"I need to solve the equation `3x + 11 = 14`. Can you help me?"',
},
],
});

let threadId = thread.id;
console.log('Created thread with Id: ' + threadId);

const run = await openai.beta.threads.runs.create(thread.id, {
assistant_id: assistantId,
additional_instructions: 'Please address the user as Jane Doe. The user has a premium account.',
});

console.log('Created run with Id: ' + run.id);

while (true) {
const result = await openai.beta.threads.runs.retrieve(thread.id, run.id);
if (result.status == 'completed') {
const messages = await openai.beta.threads.messages.list(thread.id);
for (const message of messages.getPaginatedItems()) {
console.log(message);
}
break;
} else {
console.log('Waiting for completion. Current status: ' + result.status);
await sleep(5000);
}
}
}

main();
1 change: 1 addition & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,7 @@ export namespace OpenAI {

export import Beta = API.Beta;

export import ErrorObject = API.ErrorObject;
export import FunctionDefinition = API.FunctionDefinition;
export import FunctionParameters = API.FunctionParameters;
}
Expand Down
Loading