Region based routing (per end-user)

[krrishdholakia]

Specific users need to have all their llm api calls be within eu

Allow something like this:

curl --location 'http://0.0.0.0:4000/end_user/new' \
--header 'Authorization: Bearer sk-1234' \ # 👈 Master Key
--header 'Content-Type: application/json' \
--data '{
      "end_user_id" : "ishaan-jaff-3", <- specific customer
    
    "allowed_region": "eu" <- set region for models        

             + 

    "default_model": "azure/gpt-3.5-turbo-eu" <- all calls from this user, use this model? 

}'

# return end-user object

[krrishdholakia]

Currently covers:

• Azure
• Vertex AI
• Bedrock

[krrishdholakia]

Expose - get_provider_region() helper function

[krrishdholakia]

Expose CRUD endpoints for end-users

• create
• info
• update
• delete

[krrishdholakia]

for v0 - scoping to just create + info

[krrishdholakia]

Add a pre-call check to filter models by region

[krrishdholakia]

How to get model region?

• on set_client -> make a blank request to azure deployments -> the headers return the rate limits + region info