Get openrouter.ai working with new headers: feature for extra-openai-models.yaml

[simonw]

https://openrouter.ai/ runs a paid OpenAI-compatible API for a bunch of models, including Claude 2 (which I have not yet got a real API key for).

It requires a feature that LLM doesn't support yet:

https://openrouter.ai/docs#api-keys

curl https://openrouter.ai/api/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "HTTP-Referer: $YOUR_SITE_URL" \
-H "X-Title: $YOUR_SITE_NAME" \
-d '{
  "model": "openai/gpt-4-32k", # Optional
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."}, 
    {"role": "user", "content": "Hello!"}
  ]
}'

You have to send two custom HTTP headers.

The OpenAI Python library does support that:

import openai

openai.api_base = "https://openrouter.ai/api/v1"
openai.api_key = OPENROUTER_API_KEY

response = openai.ChatCompletion.create(
    model="openai/gpt-3.5-turbo",
    messages=[...],
    headers={
        "HTTP-Referer": YOUR_SITE_URL,  # To identify your app
        "X-Title": YOUR_APP_NAME,
    },
)
reply = response.choices[0].message

[simonw]

I signed up for an API key at https://openrouter.ai/ and added this to my ~/Library/Application Support/io.datasette.llm/extra-openai-models.yaml file:

- model_id: claude
  model_name: anthropic/claude-2
  api_base: "https://openrouter.ai/api/v1"
  api_key_name: openrouter
  headers:
    HTTP-Referer: "https://llm.datasette.io/"
    X-Title: LLM

Then I ran this:

llm keys set openrouter

And pasted in my key. And now this works:

llm -m claude 'Which model are you?'

I'm Claude, an AI assistant created by Anthropic.