Use the connect protocol to generate unary AI completions #1559
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jlewi
changed the title
This was referenced Aug 19, 2024
|
@sourishkrout this is ready for review. |
Sounds good. Feel free to add me as one of the reviewers. A Slack bot reminds me twice a day about pending reviews. |
tests/extension/extension.test.ts
Outdated
| @@ -74,6 +74,8 @@ vi.mock('../../src/extension/grpc/runner/v1', () => ({})) | |||
| test('initializes all providers', async () => { | |||
| const configValues = { | |||
| binaryPath: 'bin', | |||
| // This is needed for the AIManager to initialize. | |||
| 'runme.aiBaseURL': 'http://localhost:8080/api', | |||
There was a problem hiding this comment.
I'd suggest using a different port, 8080, which is over-used. Fine inside container images, but running locally, there's a high likelihood another process bound the port locally.
There was a problem hiding this comment.
Good Idea. I changed the default to 8877.
WDYT?
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR switches from using gRPC and the timostamm client to using the connect client for unary completions. This makes the code consistent with what we are doing for the streaming completions with ghost cells.
The service definition is also published is part of Foyle's protos in the buf schema registry https://buf.build/jlewi/foyle/commits/commit/b69cd71876b144ca900524437d722a84 so we can remove the AI service defined in the runme buf package (https://github.com/stateful/runme/tree/main/pkg/api/proto/runme/ai) after this change is released.
The benefit of this change for users is that we can have a single configuration option AI Base URL for configuring the AIService. This will be used for all AI calls (unary or streaming). We can then remove the option FoyleAddress.
This will also allow us to simplify the Foyle server since we will be able to stop running gRPC and gRPC gateway. see Server Cleanup: Remove GRPC and GRPC Gateway jlewi/foyle#173