APIs for fetching and setting model and embedding providers

[3coins]

Fixes #102, #103, #105

Summary

This PR introduces the concept of ProviderConfig, that provides the values for the llm and embeddings provider and model, along with the api keys for the chat to work.

Actors

1. ProvidersActor: This actor loads and keeps a list of model and embeddings providers
2. ChatProviderActor: This actor stores the updated model provider (LLM) and provider params, which are updated when config is updated.
3. EmbeddingsProviderActor: This actor stores the updated embeddings provider and params, which are updated when config is updated.
4. ConfigActor: This actor stores the ProviderConfig object which is shared globally with other actors above, and is responsible for loading and saving the config from/to the disk.

APIs

1. GET /api/ai/providers: Returns list of model providers and models they support
2. GET /api/ai/providers/embeddings: Returns list of embeddings providers and supported model options
3. GET /api/ai/config: Returns the currently set config as ProviderConfig
4. POST /api/ai/config: Updates the config, saves to disk and updates chat provider and embeddings

Notes

1. The anthropic models don't wrap the code segments in the response in a markdown code block; I have tried updating the prompt template and hasn't seen this working yet.
   

2. The current setup is not working with certain embeddings, particularly Cohere because of the way the AskActor references the vectorstore, and uses it to get the retriever. Cohere embeddings has some non-serializable parts, which causes errors when the AskActor loads it to set the retriever but is unable to de-serialize.

   The last line here causes an exception

   index_actor = ray.get_actor(ACTOR_TYPE.LEARN.value)
   handle = index_actor.get_index.remote()
   vectorstore = ray.get(handle)

   Here is the exception:

   Traceback (most recent call last):
   File "/Users/pijain/projects/jupyter-ai-3.9/jupyter-ai/packages/jupyter-ai/jupyter_ai/actors/base.py", line 55, in process_message
       self._process_message(message)
   File "/Users/pijain/Library/Application Support/hatch/env/virtual/jupyter-ai-monorepo/X-JWbDZw/jupyter-ai-monorepo/lib/python3.9/site-packages/ray/util/tracing/tracing_helper.py", line 466, in _resume_span
       return method(self, *_args, **_kwargs)
   File "/Users/pijain/projects/jupyter-ai-3.9/jupyter-ai/packages/jupyter-ai/jupyter_ai/actors/ask.py", line 54, in _process_message
       vectorstore = ray.get(handle)
   File "/Users/pijain/Library/Application Support/hatch/env/virtual/jupyter-ai-monorepo/X-JWbDZw/jupyter-ai-monorepo/lib/python3.9/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
       return func(*args, **kwargs)
   File "/Users/pijain/Library/Application Support/hatch/env/virtual/jupyter-ai-monorepo/X-JWbDZw/jupyter-ai-monorepo/lib/python3.9/site-packages/ray/_private/worker.py", line 2309, in get
       raise value.as_instanceof_cause()
   ray.exceptions.RayTaskError(TypeError): �[36mray::LearnActor.get_index()�[39m (pid=90820, ip=127.0.0.1, repr=<jupyter_ai.actors.learn.LearnActor object at 0x138093fd0>)
   File "/Users/pijain/Library/Application Support/hatch/env/virtual/jupyter-ai-monorepo/X-JWbDZw/jupyter-ai-monorepo/lib/python3.9/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
       cp.dump(obj)
   File "/Users/pijain/Library/Application Support/hatch/env/virtual/jupyter-ai-monorepo/X-JWbDZw/jupyter-ai-monorepo/lib/python3.9/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 627, in dump
       return Pickler.dump(self, obj)
   TypeError: cannot pickle '_queue.SimpleQueue' object

3. Switching embedding providers will cause issues, since the saved index created with one provider is not portable to adding documents after switching to a new provider. In order to make this safely work for users, we have to make sure that the existing indexes are deleted when embeddings provider changes. Users should be aware of this change when they update config.

[dlqqq]

@3coins This is awesome work! Thank you very much for getting this out so quickly! 🎉 This is 95% complete; most of my suggestions are minor improvements. I had a few high-level comments in addition to the more granular ones below:

1. Can you create a separate branch (e.g. model-config) that's identical to main, and then set the target of this PR to that branch?

2. I'm tracking naming suggestions in a separate issue that should be addressed before the branch is merged into main: Model configurability naming changes #129

[dlqqq]

Can just use provider.id here 😁

[dlqqq]

Same for line 85.

[3coins]

Could not do this for the embedding provider. There is a workaround for this, but want to iterate on this in the following PRs.

[3coins]

@dlqqq
Taken care of most of the comments, issue 2 from notes is fixed now, so embedding providers work without any issues.
For issue 3 in the notes, I have added some error checking around failures when embedding provider is updated and /ask command is used, which should indicate the user to delete the index. For long term, we should not only delete the index when the embedding provider changes, but also re-index directories user has indexed with the previous provider; this needs some more work and can't be handled in this PR.