Function calling complete example

[jrobinson01]

Adjusts the function-calling example to closely follow the steps in this tutorial
Function calling is currently a bit incomplete in web-llm. The tutorial outlines how to process the function calls and feed them back into the llm in order to get the generated reply.

There were a few places in engine.ts and chatCompletion.ts where we were blocking messages that had a "tool" role so I added that role to the config and adjusted the conditionals to allow that role alongside the "user" role.

There are some typescript errors/warnings in this branch that I wasn't sure how to solve. Any guidance would be appreciated!

I was able to build web-llm but was not able to use the built files in the function calling example. I'm not certain what went wrong, but could try again and get some error logs if those would be useful.

[jrobinson01]

Here's the error I get in the browser when I try to run the function-calling example while using my local build of web-llm:

Uncaught (in promise) LinkError: WebAssembly.instantiate(): Import #6 module="wasi_snapshot_preview1" function="proc_exit": function import requires a callable

I'm trying to do so by changing package.json in function-calling like so:

"dependencies": {
    "@mlc-ai/web-llm": "file:../.."
  }

I also had the same error when trying to yarn link web-llm into the function-calling example.

[Neet-Nestor]

@CharlieFRuan May you have a look of this PR?

[CharlieFRuan]

Thanks for the contribution! Function calling in WebLLM is indeed incomplete as of now. We currently only support one round of function calling, with only the Heremes-2-Pro models.

Some more work is needed to make it fully compatible with OpenAI. For instance, an LLM's reply does not have to be a function call and can be purely natural language (which is not enabled as of now as we apply the json format whenever we use function calling). This can be fixed by using the <tool_call> token to toggle internal json formatting.

Will work on this soon! Will leave this PR open in the meantime.

[jrobinson01]

Thanks @CharlieFRuan ! The PR attempts to address a few of the issues I found to get the "current weather" example working fully. I haven't looked at what is needed to enable multi-round chat.

[CharlieFRuan]

This PR should be superseded by #527. Again, thank you for the contribution @jrobinson01 ! Apologies I did not work on top of this PR, but it offered great insights!