Skip to content

Commit

Permalink
fix: README file
Browse files Browse the repository at this point in the history
  • Loading branch information
Aris Boutselis committed May 2, 2023
1 parent e77ee83 commit dc07977
Showing 1 changed file with 27 additions and 36 deletions.
63 changes: 27 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
<picture>
<source media="(prefers-color-scheme: dark)" srcset="./images/banner-white.png" width="600px;">
<img alt="Text changing depending on mode. Light: 'So light!' Dark: 'So dark!'" src="./images/banner-black.png" width="600px;">
</picture>
<br/>
<img src="./images/banner-white.png" width="600px;" />

![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/k8sgpt-ai/k8sgpt)
![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/k8sgpt-ai/k8sgpt/release.yaml)
Expand Down Expand Up @@ -287,7 +283,32 @@ k8sgpt analyze --explain --backend azureopenai
</details>

### Running local models
</details>

To run local models, it is possible to use OpenAI compatible APIs, for instance [LocalAI](https://github.com/go-skynet/LocalAI) which uses [llama.cpp](https://github.com/ggerganov/llama.cpp) and [ggml](https://github.com/ggerganov/ggml) to run inference on consumer-grade hardware. Models supported by LocalAI for instance are Vicuna, Alpaca, LLaMA, Cerebras, GPT4ALL, GPT4ALL-J and koala.

<details>

To run local inference, you need to download the models first, for instance you can find `ggml` compatible models in [huggingface.com](https://huggingface.co/models?search=ggml) (for example vicuna, alpaca and koala).

### Start the API server

To start the API server, follow the instruction in [LocalAI](https://github.com/go-skynet/LocalAI#example-use-gpt4all-j-model).

### Run k8sgpt

To run k8sgpt, run `k8sgpt auth` with the `localai` backend:

```
k8sgpt auth --backend localai --model <model_name> --baseurl http://localhost:8080/v1
```

Now you can analyze with the `localai` backend:

```
k8sgpt analyze --explain --backend localai
```

</details>

## How does anonymization work?

Expand Down Expand Up @@ -319,36 +340,6 @@ The Kubernetes system is trying to scale a StatefulSet named fake-deployment usi

</details>


## Running local models
>>>>>>> main
To run local models, it is possible to use OpenAI compatible APIs, for instance [LocalAI](https://github.com/go-skynet/LocalAI) which uses [llama.cpp](https://github.com/ggerganov/llama.cpp) and [ggml](https://github.com/ggerganov/ggml) to run inference on consumer-grade hardware. Models supported by LocalAI for instance are Vicuna, Alpaca, LLaMA, Cerebras, GPT4ALL, GPT4ALL-J and koala.

<details>

To run local inference, you need to download the models first, for instance you can find `ggml` compatible models in [huggingface.co](https://huggingface.co/models?search=ggml) (for example vicuna, alpaca and koala).

### Start the API server

To start the API server, follow the instruction in [LocalAI](https://github.com/go-skynet/LocalAI#example-use-gpt4all-j-model).

### Run k8sgpt

To run k8sgpt, run `k8sgpt auth` with the `localai` backend:

```
k8sgpt auth --backend localai --model <model_name> --baseurl http://localhost:8080/v1
```

Now you can analyze with the `localai` backend:

```
k8sgpt analyze --explain --backend localai
```

</details>

## Configuration

<details>
Expand Down

0 comments on commit dc07977

Please sign in to comment.