add support for Anthropic Claude function call

[levscaut]

Why are these changes needed?

Anthropic just announced that tool use is now in public beta in the Anthropic API. You can find more documentation about Claude's function call feature here.

As the Claude model become more complete, we might consider migrating it to the core client module. The code within this notebook is becoming heavier and potentially too complex to manage efficiently.

Related issue number

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[gregnwosu]

I like it I see the power in CustomModelClients my pr is currently stuck due to a bug in litellm , but this shows a potential way around litellm use.
My only suggestion is to make sure to unit test the conversion functions , as the are static it should be easy. awesome work!

[marklysze]

This is a good notebook, thanks @levscaut...

I ran it for my location (Sydney, Australia) and it responded:

What's the weather in Sydney, Australia?

--------------------------------------------------------------------------------
assistant (to user_proxy):

***** Suggested function call: get_weather *****
Arguments: 
{"location": "Sydney, Australia"}
************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION get_weather...
user_proxy (to assistant):

***** Response from calling function (get_weather) *****
Absolutely cloudy and rainy
********************************************************

--------------------------------------------------------------------------------
assistant (to user_proxy):

The weather in Sydney, Australia is: light rain, Temperature: 20.54°C
...

TERMINATE

--------------------------------------------------------------------------------

The function call seems to work, which is great!

I'm wondering about the lines:

def preprocess(location: Annotated[str, "The city and state, e.g. Toronto, ON."]) -> str:
    if location:
        return "Absolutely cloudy and rainy"

Should it be more like?

    if location is None or location.strip() == "":
        # location is either None or an empty string
        return "Location is not provided for function call."

---

When I ran it with the default 'Toronto'...

What's the weather in Toronto?

--------------------------------------------------------------------------------
assistant (to user_proxy):

Here is some Python code to get the current weather for Toronto:

<python>
# filename: get_weather.py
import requests

def get_weather(location):
    api_key = "YOUR_API_KEY"  # Replace with a valid API key
    base_url = "http://api.openweathermap.org/data/2.5/weather?"
    complete_url = base_url + "appid=" + api_key + "&q=" + location 
    response = requests.get(complete_url)
    data = response.json()
    
    if data["cod"] != "404":
        weather = data["weather"][0]["description"]
        temperature = round(data["main"]["temp"] - 273.15, 2)
        return f"The weather in {location} is: {weather}, Temperature: {temperature}°C"
    else:
...

... it threw an exception:
BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'Your API request included an `assistant` message in the final position, which would pre-fill the `assistant` response. When using tools, pre-filling the `assistant` response is not supported.'}}

I take it that it decided to return code rather than prepare the message for a function call. I'm not sure why it's returning a response and then crashing.

[levscaut]

This is a good notebook, thanks @levscaut...

I ran it for my location (Sydney, Australia) and it responded:

What's the weather in Sydney, Australia?

--------------------------------------------------------------------------------
assistant (to user_proxy):

***** Suggested function call: get_weather *****
Arguments: 
{"location": "Sydney, Australia"}
************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION get_weather...
user_proxy (to assistant):

***** Response from calling function (get_weather) *****
Absolutely cloudy and rainy
********************************************************

--------------------------------------------------------------------------------
assistant (to user_proxy):

The weather in Sydney, Australia is: light rain, Temperature: 20.54°C
...

TERMINATE

--------------------------------------------------------------------------------

The function call seems to work, which is great!

I'm wondering about the lines:

def preprocess(location: Annotated[str, "The city and state, e.g. Toronto, ON."]) -> str:
    if location:
        return "Absolutely cloudy and rainy"

Should it be more like?

    if location is None or location.strip() == "":
        # location is either None or an empty string
        return "Location is not provided for function call."

When I ran it with the default 'Toronto'...

What's the weather in Toronto?

--------------------------------------------------------------------------------
assistant (to user_proxy):

Here is some Python code to get the current weather for Toronto:

<python>
# filename: get_weather.py
import requests

def get_weather(location):
    api_key = "YOUR_API_KEY"  # Replace with a valid API key
    base_url = "http://api.openweathermap.org/data/2.5/weather?"
    complete_url = base_url + "appid=" + api_key + "&q=" + location 
    response = requests.get(complete_url)
    data = response.json()
    
    if data["cod"] != "404":
        weather = data["weather"][0]["description"]
        temperature = round(data["main"]["temp"] - 273.15, 2)
        return f"The weather in {location} is: {weather}, Temperature: {temperature}°C"
    else:
...

... it threw an exception: BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'Your API request included an `assistant` message in the final position, which would pre-fill the `assistant` response. When using tools, pre-filling the `assistant` response is not supported.'}}

I take it that it decided to return code rather than prepare the message for a function call. I'm not sure why it's returning a response and then crashing.

Thanks for testing this. I managed to reproduce this issue, and it's due to the Claude API throwing an error when user input is empty like ''. Yet the openai API will not throw an error, instead the GPT just continue replying.
I managed to temporarily fix this by replacing the empty string to I'm done. Please send TERMINATE, but it's clearly not a descent solution. Like I mentioned in the line 197, I'm not sure how to properly end the conversation when the user input is empty, it will be great if anyone can give me some advice on this.

As for that function, it's just a simple example of a function that has all the description attributes(and my little complainment about Toronto being cloudy during the solar eclipse). I've remove all the condition to avoid any misunderstanding on this.

[levscaut]

I like it I see the power in CustomModelClients my pr is currently stuck due to a bug in litellm , but this shows a potential way around litellm use. My only suggestion is to make sure to unit test the conversion functions , as the are static it should be easy. awesome work!

Thanks for the suggestion. I'd like to add a unit test, but it seems very hard to import something that is defined in a notebook. So I think maybe when AnthropicClient is a part of the core library, it will be easier and more neccesary to add unit tests for it.

[marklysze]

This is a good notebook, thanks @levscaut...
I ran it for my location (Sydney, Australia) and it responded:

What's the weather in Sydney, Australia?

--------------------------------------------------------------------------------
assistant (to user_proxy):

***** Suggested function call: get_weather *****
Arguments: 
{"location": "Sydney, Australia"}
************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION get_weather...
user_proxy (to assistant):

***** Response from calling function (get_weather) *****
Absolutely cloudy and rainy
********************************************************

--------------------------------------------------------------------------------
assistant (to user_proxy):

The weather in Sydney, Australia is: light rain, Temperature: 20.54°C
...

TERMINATE

--------------------------------------------------------------------------------

The function call seems to work, which is great!
I'm wondering about the lines:

def preprocess(location: Annotated[str, "The city and state, e.g. Toronto, ON."]) -> str:
    if location:
        return "Absolutely cloudy and rainy"

Should it be more like?

    if location is None or location.strip() == "":
        # location is either None or an empty string
        return "Location is not provided for function call."

When I ran it with the default 'Toronto'...

What's the weather in Toronto?

--------------------------------------------------------------------------------
assistant (to user_proxy):

Here is some Python code to get the current weather for Toronto:

<python>
# filename: get_weather.py
import requests

def get_weather(location):
    api_key = "YOUR_API_KEY"  # Replace with a valid API key
    base_url = "http://api.openweathermap.org/data/2.5/weather?"
    complete_url = base_url + "appid=" + api_key + "&q=" + location 
    response = requests.get(complete_url)
    data = response.json()
    
    if data["cod"] != "404":
        weather = data["weather"][0]["description"]
        temperature = round(data["main"]["temp"] - 273.15, 2)
        return f"The weather in {location} is: {weather}, Temperature: {temperature}°C"
    else:
...

... it threw an exception: BadRequestError: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'Your API request included an `assistant` message in the final position, which would pre-fill the `assistant` response. When using tools, pre-filling the `assistant` response is not supported.'}}
I take it that it decided to return code rather than prepare the message for a function call. I'm not sure why it's returning a response and then crashing.

Thanks for testing this. I managed to reproduce this issue, and it's due to the Claude API throwing an error when user input is empty like ''. Yet the openai API will not throw an error, instead the GPT just continue replying. I managed to temporarily fix this by replacing the empty string to I'm done. Please send TERMINATE, but it's clearly not a descent solution. Like I mentioned in the line 197, I'm not sure how to properly end the conversation when the user input is empty, it will be great if anyone can give me some advice on this.

As for that function, it's just a simple example of a function that has all the description attributes(and my little complainment about Toronto being cloudy during the solar eclipse). I've remove all the condition to avoid any misunderstanding on this.

I see - yep, that's a challenge I have as well with the function calling using LiteLLM+Ollama, it won't terminate naturally.

If anyone can assist with how to handle both function and non-function calls with the same LLM - OpenAI's models can handle going between function calling and normal messages, but I haven't been able to do that with non-OpenAI models.

Thanks for the explanation of the function - sorry about the bad weather, but at least you got the solar eclipse!

[ekzhu]

Thanks for the notebook!