Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Runtime for tool calling #1163

Merged
merged 52 commits into from
Nov 22, 2024
Merged

feat: Add Runtime for tool calling #1163

merged 52 commits into from
Nov 22, 2024

Conversation

liuxukun2000
Copy link
Collaborator

@liuxukun2000 liuxukun2000 commented Nov 8, 2024

Description

Add runtime class for tool calling: #1014

  • Docker Runtime
  • Remote HTTP Runtime
  • LLM Guard Runtime

Motivation and Context

Issue: #1014

  • I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds core functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (update in the documentation)
  • Example (update in the folder of example)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

  • I have read the CONTRIBUTION guide. (required)
  • My change requires a change to the documentation.
  • I have updated the tests accordingly. (required for a bug fix or a new feature)
  • I have updated the documentation accordingly.

Copy link
Member

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @liuxukun2000 ! The core code looks great now, I added one more commit for small docstring and code format polish here: 4c4d4ac , feel free to check. The last thing is the unit test to be added, could you add unit test under test folder? Thanks a lot!

camel/runtime/api.py Show resolved Hide resolved
examples/runtime/remote_http_runtime.py Outdated Show resolved Hide resolved
@Asher-hss
Copy link
Collaborator

Hi, Xukun,

As a framework, I think Camel shouldn't rely on starting an HTTP service for running tools. If the HTTP port is exposed, it introduces additional security concerns that need to be addressed.

Would it be possible to directly start a Docker container, execute the corresponding tool script (.py), and destroy the container immediately after execution?

@liuxukun2000
Copy link
Collaborator Author

Hi @Asher-hss , thank you for your suggestion and for considering the potential security implications of starting an HTTP service. However, I believe HTTP security concerns are not something the framework itself should manage, as long as the service is used properly and securely. HTTP servers are widely adopted for implementing function calls due to their versatility and ease of integration.
Additionally, using an HTTP server ensures both efficiency and greater flexibility, which is particularly critical in benchmarks where frequent function calls are required. It also allows us to maintain the context of function calls, which would be challenging to achieve with your proposed approach of using ephemeral Docker containers.
Let me know if you'd like to discuss this further or have additional thoughts!

@Asher-hss
Copy link
Collaborator

Asher-hss commented Nov 19, 2024

@liuxukun2000 Hi xukun, I really appreciate the consideration for flexibility—it’s definitely an important factor. That said, I’m also thinking about starting a service inherently aims to create a long-running environment, which might not fully align with the original goals we had in mind for designing this runtime. Then, could you share an example scenario where maintaining the function call context would be particularly useful? Lastly, I’d love to hear your thoughts on what specific use cases you see for adopting an HTTP runtime.

@Wendong-Fan Wendong-Fan merged commit 2f8881d into master Nov 22, 2024
5 of 6 checks passed
@Wendong-Fan Wendong-Fan deleted the feat/runtime branch November 22, 2024 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

5 participants