using embeddings

[lalligagger]

In relation to #110 and #92, adding a simple embedding search & context injection would be helpful to maximize use of the context window. OpenAI released some minimal example notebooks (1, 2) today that pull embeddings from a .csv and store in memory as a dataframe.

Any thoughts on supporting basic embeddings search through shell-gpt?

[lalligagger]

Here's my current make_prompt.py which, even without hitting token limits, seems easy to "overflow" the context. I.e., if too much irrelevant info is added it seems to reduce the effectiveness of the relevant added context.

So I think this could all be handled in a JSON config per-role, with a list of pointers to embeddings to use. At least in my case, the reason I'm using shell-gpt vs. langchain etc. is the simplicity... CSV embeddings would be fine for the foreseeable future. Assuming latest understanding of #92 functionality, I'd say:

1. The role.json has an optional field 'embeddings'
2. If embeddings are supplied, a search is performed using current prompt (+ history?) and top hits are appended to {added_context}, which get injected ahead of the user prompt up to the token limit
3. If no embeddings are supplied, nothing changes
4. A given role can reference one or more embeddings stores, that will encourage smaller stores - I see this providing a ton of functionality with even modestly sized CSV stores

Anything I'm glossing over? I'd be happy to take a crack at this once the user-defined roles are available in a stable branch if not released.

[TheR1D]

Thanks for sharing your idea about improving ShellGPT! It seems like the embedding part you mentioned might be more useful for your own needs rather than most sgpt users. Main goal of ShellGPT is to help devs with code, shell commands, and other dev stuff. But don't worry, I'm working on adding system roles, and that might give you a starting point to build something that works for you. Once we have user-defined roles released, you can definitely give it a shot and share your results.

P.S. I think fine-tuning would be better choice for your specific use case.

[lalligagger]

Sounds good and no pressure of course! In my case I would be pointing to specific code docs which improve the shell-gpt output. I get runnable commands with the extra context but not without.

Have done some fine tuning towards this goal and it didn't work nearly as well. Here is the relevant snippet from OpenAI in the cookbook:

Although fine-tuning can feel like the more natural option—training on data is how GPT learned all of its other knowledge, after all—we generally do not recommend it as a way to teach the model knowledge. Fine-tuning is better suited to teaching specialized tasks or styles, and is less reliable for factual recall.

[lpurdy01]

I have been playing with the extended context capabilities of LangChain agents + embeddings vectorstore in the context of information finding chatbots on Discord and Slack and they have proven themselves to be very powerful. The ability to give a model access to the entire documentation for a library and some .pdfs of the scientific theory to have it act with all of that grabbable is really powerful.

Unfortunately, in order to use context / vectorstore properly shell_gpt would need to adopt an agent architecture. This is necessary for the AI to have 'thoughts' about what it should query its database for and collect the relevant bits of information into a memory.

Shell GPT's REPL mode is close to this, but a lot of the backend would have to be replaced with stuff that either replicates or implements langchain stuff. Then shell gpt would basically become an Auto-GPT clone. (Which would be nice because Auto-GPT's terminal interface is terrible.)

I think the best thing @TheR1D has done is create an awesome UI.