Initial plugin

[simonw]

Goal is to allow people to llm install llm-openrouter and set an API key and instantly get access to the full set of openrouter.ai models - as described in this JSON: https://openrouter.ai/api/v1/models

This works in LLM main right now using extra-openai-models.yml but I think it could be more usable as a separate plugin:

• Get openrouter.ai working with new headers: feature for extra-openai-models.yaml llm#165

Documented here: https://llm.datasette.io/en/latest/other-models.html#extra-http-headers

- model_id: claude
  model_name: anthropic/claude-2
  api_base: "https://openrouter.ai/api/v1"
  api_key_name: openrouter
  headers:
    HTTP-Referer: "https://llm.datasette.io/"
    X-Title: LLM

[simonw]

Installing the plugin will start caching a local copy of https://openrouter.ai/api/v1/models - as seen in llm-gpt4all: https://github.com/simonw/llm-gpt4all/blob/0046e2bf5d0a9c369b804d7125a1ab50bd5878f1/llm_gpt4all.py#L26-L31

The JSON format looks like this:

{
  "data": [
    {
      "id": "openai/gpt-3.5-turbo",
      "pricing": {
        "prompt": "0.0000015",
        "completion": "0.000002"
      },
      "context_length": 4095,
      "per_request_limits": {
        "prompt_tokens": "2871318",
        "completion_tokens": "2153488"
      }
    },
    {
      "id": "openai/gpt-3.5-turbo-0301",
      "pricing": {
        "prompt": "0.0000015",
        "completion": "0.000002"
      },
      "context_length": 4095,
      "per_request_limits": {
        "prompt_tokens": "2871318",
        "completion_tokens": "2153488"
      }
    },

I'm going to set model IDs of openrouter/openai/gpt-3.5-turbo-0301 - a bit wordy, but makes it clear that they differ from the same models from other plugins.

Users can set aliases if they want shorter model indicators.

[simonw]

This is going to work by importing Chat from llm.default_plugins.openai_models.

At some point soon I'm going to move the OpenAI stuff out of LLM core and into a plugin, at which point this plugin will need to depend on it.

[simonw]

... or maybe I should do that first.