Skip to content

Commit

Permalink
feat: validate backend name
Browse files Browse the repository at this point in the history
Signed-off-by: Aris Boutselis <aris.boutselis@senseon.io>
  • Loading branch information
Aris Boutselis committed Apr 27, 2023
1 parent 04bbb9c commit c75170a
Show file tree
Hide file tree
Showing 4 changed files with 66 additions and 18 deletions.
27 changes: 24 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,8 +313,31 @@ _Analysis with serve mode_
```
curl -X GET "http://localhost:8080/analyze?namespace=k8sgpt&explain=false"
```
</details>

## Additional AI providers

### Azure OpenAI
<em>Prerequisites:</em> an Azure OpenAI deployment is needed, please visit MS official [documentation](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource) to create your own.

To authenticate with k8sgpt, you will need the Azure OpenAI endpoint of your tenant `"https://your Azure OpenAI Endpoint"`, the api key to access your deployment, the deployment name of your model and the model name itself.
<details>

### Run k8sgpt
To run k8sgpt, run `k8sgpt auth` with the `azureopenai` backend:
```
k8sgpt auth --backend azureopenai --baseurl https://<your Azure OpenAI endpoint> --engine <deployment_name> --model <model_name>
```
Lastly, enter your Azure API key, after the prompt.

## Running local models
Now you are ready to analyze with the azure openai backend:
```
k8sgpt analyze --explain --backend azureopenai
```

</details>

### Running local models

To run local models, it is possible to use OpenAI compatible APIs, for instance [LocalAI](https://github.com/go-skynet/LocalAI) which uses [llama.cpp](https://github.com/ggerganov/llama.cpp) and [ggml](https://github.com/ggerganov/ggml) to run inference on consumer-grade hardware. Models supported by LocalAI for instance are Vicuna, Alpaca, LLaMA, Cerebras, GPT4ALL, GPT4ALL-J and koala.

Expand All @@ -334,8 +357,6 @@ To run k8sgpt, run `k8sgpt auth` with the `localai` backend:
k8sgpt auth --backend localai --model <model_name> --baseurl http://localhost:8080/v1
```

When being asked for an API key, just press enter.

Now you can analyze with the `localai` backend:

```
Expand Down
22 changes: 19 additions & 3 deletions cmd/auth/auth.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,13 @@ var AuthCmd = &cobra.Command{
Use: "auth",
Short: "Authenticate with your chosen backend",
Long: `Provide the necessary credentials to authenticate with your chosen backend.`,
PreRun: func(cmd *cobra.Command, args []string) {
backend, _ := cmd.Flags().GetString("backend")
if backend == "azureopenai" {
cmd.MarkFlagRequired("engine")
cmd.MarkFlagRequired("baseurl")
}
},
Run: func(cmd *cobra.Command, args []string) {

// get ai configuration
Expand All @@ -58,9 +65,18 @@ var AuthCmd = &cobra.Command{
}
}

// check if backend is not empty
if backend == "" {
color.Red("Error: Backend AI cannot be empty.")
validBackend := func(validBackends []string, backend string) bool {
for _, b := range validBackends {
if b == backend {
return true
}
}
return false
}

// check if backend is not empty and a valid value
if backend == "" || !validBackend(ai.Backends, backend) {
color.Red("Error: Backend AI cannot be empty and accepted values are '%v'", strings.Join(ai.Backends, ", "))
os.Exit(1)
}

Expand Down
3 changes: 2 additions & 1 deletion cmd/serve/serve.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,14 @@ var ServeCmd = &cobra.Command{
engine := os.Getenv("K8SGPT_ENGINE")
// If the envs are set, alocate in place to the aiProvider
// else exit with error
envIsSet := backend != "" || password != "" || model != "" || baseURL != "" || engine != ""
envIsSet := backend != "" || password != "" || model != "" || baseURL != ""
if envIsSet {
aiProvider = &ai.AIProvider{
Name: backend,
Password: password,
Model: model,
BaseURL: baseURL,
Engine: engine,
}

configAI.Providers = append(configAI.Providers, *aiProvider)
Expand Down
32 changes: 21 additions & 11 deletions pkg/ai/iai.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,21 @@ import (
"github.com/k8sgpt-ai/k8sgpt/pkg/cache"
)

var (
clients = []IAI{
&OpenAIClient{},
&AzureAIClient{},
&LocalAIClient{},
&NoOpAIClient{},
}
Backends = []string{
"openai",
"localai",
"azureopenai",
"noopai",
}
)

type IAI interface {
Configure(config IAIConfig, language string) error
GetCompletion(ctx context.Context, prompt string) (string, error)
Expand All @@ -34,18 +49,13 @@ type IAIConfig interface {
}

func NewClient(provider string) IAI {
switch provider {
case "openai":
return &OpenAIClient{}
case "azureopenai":
return &AzureAIClient{}
case "localai":
return &LocalAIClient{}
case "noopai":
return &NoOpAIClient{}
default:
return &OpenAIClient{}
for _, c := range clients {
if provider == c.GetName() {
return c
}
}
// default client
return &OpenAIClient{}
}

type AIConfiguration struct {
Expand Down

0 comments on commit c75170a

Please sign in to comment.